loci

8.12

loci🔗ℹ

Locus (pl. loci): the place where something is situated or occurs; - Merriam-Webster Dictionary

Loci enables the development of parallel programs that can take advantage of machines with multiple processors, cores or hardware threads... but this time for real!

Currently, the loci library only works on Linux and MacOS due to the use of unix channels for communication between loci.

This library attempts to follow the places API very closely. When this is not the case, feel free to report a bug. As opposed to places from the standard library, loci use OS processes instead of OS threads to allow the parallelization of multiple tasks. Up until 16 simultaneous places there’s very little different between loci and places but once you go into 32, 64, 144, 256, etc. core machines then places will break down and stop working properly due to memory allocation inter-thread locking.

1 API Documentation🔗ℹ

(require loci)

package: loci

procedure
(locus-enabled?) → boolean?

Returns #t. It exists for compatibility with the place API. Loci are always enabled.

procedure
(locus? v) → boolean?
v : any/c

Returns #t if v is a locus descriptor value, #f otherwise. Every locus descriptor is also a locus channel.

procedure
(dynamic-locus module-path start-name) → locus?
module-path : (or/c module-path? path?)
start-name : symbol?

Creates a locus to run the procedure that is identified by module-path and start-name. The result is a locus descriptor value that represents the new parallel task; the locus descriptor is returned immediatelly. The locus descriptor is also a locus channel that permits communication with the locus.

The module indicated by module-path must export a function with the name start-name. The function must accept a single argument, which is a locus channel that corresponds to the other end of communication for the locus descriptor returned by locus.

When the locus is created, the initial exit handler terminates the locus, using the argument to the exit handler as the locus’ completion value. Use (exit v) to immediatelly terminate a locus with the completion value v. Since a completion value is limited to an exact integer between 0 and 255, any other value for v is converted to 0.

If the function indicated by module-path and start-name returns, the the locus terminates with the completion value 0.

In the created locus, the current-input-port parameter is set to an empty input port, while the values of the current-output-port and current-error-port parameters are connected to the ports in the creating locus.

syntax
(locus id body ...+)

{ Creates a locus that evaluates body expressions with id bound to a locus channel. The bodys close only over id plus the top-level bindings of the enclosing module, because the bodys are lifted to a submodule. The result of locus is a locus descriptor, like the result of dynamic-locus.

The generated submodule has the name locus-body-n for an integer n, and the submodule exports a main function that takes a locus channel for the new locus. The submodule is not intended for use, however, except by the expansion of the locus form.

The locus binding is protected in the same way as dynamic-locus. }

syntax
(locus/context id body ...+)

Like locus, but body ... may have free lexical variables, which are automatically sent to the newly-created locus. Note that these variables must have values accepted by locus-message-allowed?, otherwise an exn:fail:contract exception is raised.

procedure
(locus-wait l) → exact-integer?
l : locus?

Returns the completion value of the locus indicated by l, blocking until the locus has terminated.

If any pumping threads were created to connect a non-file-stream port to the ports in the locus for l (see dynamic-locus), locus-wait returns only when the pumping threads have completed.

procedure
(locus-dead-evt l) → evt?
l : locus?

Returns a synchronizable event (see (part "sync")) that is ready for synchronization if and only if l has terminated. The synchronization result of a locus-dead event is the locus-dead event itself.

If any pumping threads were created to connect a non-file-stream port to the ports in the locus for l (see dynamic-locus), the event returned by locus-dead-evt may become ready even if a pumping thread is still running.

procedure
(locus-kill l) → void?
l : locus?

Immediately terminates the locus, setting the locus’ completion value to 1 if the locus does not have a completion value already.

2 Motivation🔗ℹ

Given a problem that requires parallelism, where you use all the available cores in your machine, the Racket answer as been places. I happily used places for a long time until I started stressing tests by getting larger and larger machines (up to 144 cores) and starting 144 places simultaneously. I noticed that once I exceeded 12 cores something started to go wrong, in particular the cores were idle most of the time and there was a lot of kernel locking taking place. This triggered me sending a message to the Racket mailing list for help.