Dataspace Programming with Syndicate
1 Overview
2 Programming Syndicate Actors with Facets
2.1 Script Actions:   Starting and Stopping Actors and Facets
spawn
react
stop-facet
stop-current-facet
current-facet-id
send!
2.2 Installing Endpoints
field
assert
on
during
during/  spawn
stop-when
2.3 Handling Facet Startup and Shutdown
on-start
on-stop
2.4 Streaming Query Fields
define/  query-set
define/  query-hash
define/  query-value
define/  query-count
2.5 Generalizing Dataflow
begin/  dataflow
define/  dataflow
2.6 Generalizing Actor-Internal Communication
2.7 Nesting Dataspaces
dataspace
3 #lang syndicate Programs
4 Interacting with the Outside World
5 Actors with an Agenda
6 Odds and Ends
assert!
retract!
state
until
forever
7.9

Dataspace Programming with Syndicate

 (require syndicate/actor) package: syndicate

1 Overview

Syndicate is an actor language where all communication occurs through a tightly controlled shared memory, dubbed the dataspace. The values in the dataspace are called assertions, representing the information that the actors in the system are currently sharing with each other. Assertions are read-only and owned by the actor that entered them into the dataspace. Only the originating actor has permission to withdraw an assertion. Assertions are linked to the lifetime of their actor, and are withdrawn from the dataspace when that actor exits, either normally or via exception.

To respond to assertions in the dataspace, an actor expresses an interest in the shape of assertions it wishes to receive. An interest is an assertion constructed with observe and wildcards where the actor wishes to receive any matching assertion. When an actor makes an assertion of interest, the dataspace dispatches the set of all matching assertions to that actor. Moreover, the dataspace keeps the actor up-to-date, informing it when a new assertion appears matching its interest, as well as when a matching assertion disappears from the dataspace. Thus, dataspaces implement a form of publish/subscribe communication.

In addition to assertions, dataspaces support instantaneous message broadcast. At the time a message is sent, all actors with a matching interest receive notification.

In response to an event, that is, a message broadcast or assertion appearance/disappearance matching an expressed interest, a Syndicate actor may take any of the following actions:
  • Updating its internal state;

  • Making or withdrawing assertions;

  • Sending broadcast messages;

  • Spawning additional actors;

  • Exiting;

  • Or any combination of these.

Thus, each individual Syndicate actor has three fudamental concerns:

Each concern is addressed by a separate language construct, which are collectively dubbed endpoints:

Endpoints are tied together via dataflow. Thus, the assertions of an actor automatically reflect the current value of its fields.

Implementing an actor’s role in a particular conversation typically involves some combination of these behaviors; a facet is a collection of related endpoints constituting the actor’s participation in a particular conversation.

Each actor starts with a single facet, and may add new facets or terminate current ones in response to events. The facets of an actor form a tree, where the parent of a particular facet is the facet in which it was created. The tree structure affects facet shutdown; terminating a facet also terminates all of its descendants.

To recap: an actor is a tree of facets, each of which comprises of a collection of endpoints.

2 Programming Syndicate Actors with Facets

Code within Syndicate actors executes in one of two contexts:
  • The endpoint-installation context occurs during the creation of a new facet, when all of its endpoints are created.

  • The script context occurs during the execution of event handlers, and permits creating/terminating facets, sending messages, and spawning actors.

The actions permitted by the two contexts are mutually exclusive, and trying to perform an action in the wrong context will give rise to a run-time error.

Within the following descriptions, we use EI as a shorthand for expressions that execute in an endpoint-installation context and S for expressions in a script context.

2.1 Script Actions: Starting and Stopping Actors and Facets

syntax

(spawn maybe-name
       maybe-assertions
       maybe-linkage
       EI ...+)
 
maybe-name = 
  | #:name name-expr
     
maybe-assertions = 
  | #:assertions assertion-expr
  | #:assertions* assertions-expr
     
maybe-linkage = 
  | #:linkage [linkage-expr ...]
 
  assertion-expr : any/c
  assertions-expr : trie?
Spawn an actor with a single inital facet whose endpoints are installed by EI. That is, there is an implicit react around EI .... Allowed within a script and module-top-level.

An optionally provided name-expr is associated with the created actor. The name is only used for error and log messages, thus is mainly useful for debugging.

The actor may optionally be given some initial assertions, which come into being at the same time as the actor. (Otherwise, the actor spawns, then boots its initial facet(s), then establishes any ensuing assertions.) When assertion-expr is provided, the actors initial assertions are the result of interpreting the expression as a trie pattern, with ? giving rise to infinte sets. On the other hand, assertions-expr may be used to specify an entire set of initial assertions as an arbitrary trie.

The optional linkage-exprs are executed during facet startup; your simple documentation author is not sure why they are useful, as opposed to just putting them in the body of the spawn.

syntax

(react EI ...+)

Create a new facet in the current actor whose endpoints are the result of executing EI .... Allowed within a script.

syntax

(stop-facet fid S ...)

 
  fid : facet-id?
Terminate the facet with ID fid, as well as all of its children. Allowed within a script.

The optional script actions S ... function like a continuation. They run after the facet and all of its children finish shutting down, i.e. after all stop handlers have executed. Moreover, S ... runs in the context of the parent of fid. Thus, any facet created by the script survives termination and will have fid’s parent as its own parent.

Note that fid must be an ancestor of the current facet.

syntax

(stop-current-facet S ...)

Stop the currently running facet; equivalent to
.

Allowed within a script.

procedure

(current-facet-id)  facet-id?

Retrieves the ID of the currently running facet.

procedure

(send! v)  void?

  v : any/c
Sends a message with body v.

Allowed within a script.

2.2 Installing Endpoints

syntax

(field [x init-expr maybe-contract] ...+)

 
maybe-contract = 
  | #:contract in
  | #:contract in out
Define fields for the current facet. Each x is bound to a handle function: calling (x) retrieves the current value, while (x v) sets the field to v.

Fields may optionally have a contract; the in contract is applied when writing to a field, while the (optional) out contract applies when reading a value from a field.

Allowed within an endpoint installation context.

syntax

(assert maybe-pred exp)

 
maybe-pred = 
  | #:when pred
 
  pred : boolean?
Make the assertion exp while the enclosing facet is active. Publishing the assertion can be made conditional on a boolean expression by supplying a #:when predicate, in which case the assertion is made only when pred evaluates to a truthy value.

If the expression exp refers to any fields, then the assertion created by the endpoint is automatically kept up-to-date each time any of those fields is updated. More specifically, the will issue a patch retracting the assertion of the previous value, replacing it with the results of reevaluating exp with the current values of each field.

Allowed within an endpoint installation context.

syntax

(on maybe-pred event-description
    S ...+)
 
maybe-pred = 
  | #:when pred
     
event-description = (message pattern)
  | (asserted pattern)
  | (retracted pattern)
     
pattern = _
  | $id
  | ($ id pattern)
  | (? pred pattern)
  | (ctor pattern ...)
  | expr
 
  pred : boolean?
Creates an event handler endpoint that responds to the event specified by event-description. Executes the body S ... for each matching event, with any pattern variables bound to their matched value.

The actor will make an assertion of interest in events that could match event-description. Like with assert, the interest will be refreshed any time a field referenced within the event-description pattern changes.

The event handler can optionally be made conditional on a boolean expression by supplying a #:when predicate, in which case the endpoint only reacts to events, and only expresses the corresponding assertion of interest, when pred evaluates to a truthy value.

Allowed within an endpoint installation context.

Event descriptions have one of the following forms:
  • (message pattern) activates when a message is received with a body matching pat.

  • (asserted pattern) activates when a patch is received with an added assertion matching pattern. Additionally, if the actor has already received a patch with matching assertions, which can occur if multiple facets in a single actor have overlapping interests, then the endpoint will match those assertions upon facet start up.

  • (retracted pat) is similar to asserted, but for assertions withdrawn in a patch.

While patterns have the following meanings:
  • _ matches anything.

  • $id matches anything and binds the value to id.

  • ($ id pattern) matches values that match pattern and binds the value to id.

  • (? pred pattern) matches values where (pred val) is not #f and that match pattern.

  • (ctor pat ...) matches values built by applying the constructor ctor to values matching pat .... ctor is usually a struct name.

  • expr patterns match values that are equal? to expr.

syntax

(during pattern EI ...+)

Engage in behavior for the duration of a matching assertion. Roughly equivalent to:

(on (asserted pattern)
  (react
    EI ...
    (on (retracted inst-pattern)
        (stop-current-facet))))

where inst-pattern is the pattern with variables instantiated to their matching values.

Allowed within an endpoint installation context.

syntax

(during/spawn pattern
               maybe-actor-wrapper
               maybe-name
               maybe-assertions
               maybe-parent-let
               maybe-on-crash
               EI ...)
 
maybe-actor-wrapper = 
  | #:spawn wrapper-stx
     
maybe-parent-let = 
  | #:let [x expr] ...
     
maybe-on-crash = 
  | #:on-crash on-crash-expr
Like during, but in addition to creating a new facet for each matching assertion, spawns a new actor. The difference is primarily relevant for error propagation; an exception inside during causes the entire actor to crash, while an exception inside during/spawn crashes only the newly spawned actor.

The assertion triggering the during/spawn may disappear before the spawned actor boots, in which case it fails to see the retraction event. To avoid potential glitches, the spawning actor maintains an assertion that lets the spawned actor know whether the originial assertion still exists.

The maybe-name and maybe-assertions have the same meaning they do for spawn, applied to the newly spawned actor.

The wrapper-stx serves as an interposition point; it may be provided to change the meaning of "spawning an actor" in response to an assertion. By default, it is #'spawn.

The optional #:let clauses can be used to read the values of fields in the spawning actor so that they can be used in the spawned actor. Otherwise, the spawned actor has no access to the parent’s fields, and trying to read or write to such a field will cause a runtime error.

The on-crash-expr provides a hook for script actions that can be performed in the spawning actor if the spawned actor crashes.

Allowed within an endpoint installation context.

syntax

(stop-when maybe-pred event-description S ...)

 
maybe-pred = 
  | #:when pred
 
  pred : boolean?
Stop the current facet when an event matching event-description occurs. Roughly equivalent to
(on event-description
    (stop-current-facet S ...))

Allowed within an endpoint installation context.

2.3 Handling Facet Startup and Shutdown

In addition to external events, such as assertion (dis)appearance and message broadcast, facets can react to their own startup and shutdown. This provides a handy way to perform initialization, cleanup, as well as setting up and tearing down resources.

syntax

(on-start S ...)

Perform the script actions S ... upon facet startup.

Allowed within an endpoint installation context.

syntax

(on-stop S ...)

Perform the script actions S ... upon facet shutdown.

The script S ... differs from that of stop-facet in that it executes in the context of the terminating facet, not its parent. Thus, any facets created in S ... will start up and then immediately shut down.

Allowed within an endpoint installation context.

Note that a single facet may have any number of on-start and on-stop handlers, which do not compete with each other. That is, each on-start handler runs during facet startup and, likewise, each on-stop during facet shutdown.

2.4 Streaming Query Fields

Syndicate actors often aggregate information about current assertions as part of their local state, that is, in a field. Since these patterns are exceedingly common, Syndicate provides a number of forms for defining fields that behave as streaming queries over the assertions in the dataspace.

syntax

(define/query-set name pattern expr maybe-on-add maybe-on-remove)

 
maybe-on-add = 
  | #:on-add on-add-expr
     
maybe-on-remove = 
  | #:on-remove on-remove-expr
Define a field called name that is the set of values extracted from assertions matching pattern. Each value is extracted from a matching assertion by evaluating expr, which may refer to variables bound by pattern.

The query set expands to roughly the following code:
(begin
  (field [name (set)])
  (on (asserted pattern)
      (name (set-add (name) expr)))
  (on (retracted pattern)
      (name (set-remove (name) expr))))

The optional on-add-expr is performed inside the on asserted handler, while on-remove-expr runs in the on retracted handler.

Allowed within an endpoint installation context.

syntax

(define/query-hash name pattern key-expr value-expr
                     maybe-on-add
                     maybe-on-remove)
 
maybe-on-add = 
  | #:on-add on-add-expr
     
maybe-on-remove = 
  | #:on-remove on-remove-expr
Define a field called name that is a hash based on assertions matching pattern. Each matching assertion establishes a key in the hash based on key-expr whose value is the result of value-expr, with each expression referring to variables bound by pattern. When a matching assertion disappears from the dataspace, the associated key is removed from the hash.

The optional maybe-on-add and maybe-on-expr behave the same way they do for define/query-set.

Allowed within an endpoint installation context.

syntax

(define/query-value name absent-expr pattern expr
                      maybe-on-add
                      maybe-on-remove)
 
maybe-on-add = 
  | #:on-add on-add-expr
     
maybe-on-remove = 
  | #:on-remove on-remove-expr
Define a field called name whose value is based on the presence of an assertion matching pattern in the dataspace. When such an assertion is present, the value of the name field is the result of evaluating expr, which may refer to pattern. When no such assertion exists, including initially, the value of name is the result of absent-expr.

The optional maybe-on-add and maybe-on-expr behave the same way they do for define/query-set.

Allowed within an endpoint installation context.

syntax

(define/query-count name pattern key-expr
                      maybe-on-add
                      maybe-on-remove)
 
maybe-on-add = 
  | #:on-add on-add-expr
     
maybe-on-remove = 
  | #:on-remove on-remove-expr
Define a field called name whose value is a hash counting occurrences of matching assertions in the dataspace. More precisely, for each assertion pattern, evaluating key-expr determines a key in the hash; the value for that key is incremented when the assertion appears and decremented when it disappears. When the count associated with a particular key falls to 0, that key is removed from the hash.

The optional maybe-on-add and maybe-on-expr behave the same way they do for define/query-set.

Allowed within an endpoint installation context.

2.5 Generalizing Dataflow

The dataflow mechanism that automatically refreshes assert endpoints when a referenced field changes may be used to react to local state updates in arbitrary ways using begin/dataflow.

syntax

(begin/dataflow S ...+)

Evaluate and perform the script actions S ... during facet startup, and then again each time a field referenced by the script updates.

Conceptually, begin/dataflow may be thought of as an event handler endpoint in the vein of on, where the event of interest is update of local state.

Allowed within an endpoint installation context.

syntax

(define/dataflow name expr maybe-default)

 
maybe-default = 
  | #:default default-expr
Define a field named name, whose value is reevaluated to the result of expr each time any referenced field changes.

The value of name is either #f or, if provided, default-expr. This initial value is observable for a short time during facet startup.

Note that when a field referenced by expr changes, there may be some time before name refreshes, during which "stale" values may be read from the field.

Allowed within an endpoint installation context.

2.6 Generalizing Actor-Internal Communication

Talk about internal assertions and messages.

2.7 Nesting Dataspaces

Nested dataspaces, inbound and outbound assertions, quit-datapace.

syntax

(dataspace S ...)

Spawns a dataspace as a child of the dataspace enclosing the executing actor. The new dataspace executes each action S.

Allowed within a script.

3 #lang syndicate Programs

In a #lang syndicate program, the results of top-level expressions define the initial group of actors in the dataspace. That is, evaluating spawn or dataspace in the context of the module top-level adds that actor specification to the initial dataspace of the program. For example, a module such as:

0 #lang syndicate
1  
2 (define (spawn-fun)
3   (spawn ...))
4  
5 (spawn ...)
6  
7 (spawn-fun)

launches a syndicate program with two initial actors, one the result of the spawn expression on line 5 and one the result of evaluating the spawn expresion on line 3 during the course of calling spawn-fun on line 7.

The initial dataspace is referred to as the ground dataspace, and it plays a special role in Syndicate programming; see below.

4 Interacting with the Outside World

ground dataspace, drivers, etc.

5 Actors with an Agenda

Here we talk about spawn* and react/suspend.

6 Odds and Ends

procedure

(assert! v [#:meta-level level])  void?

  v : any/c
  level : natural-number/c = 0
Asserts the value of v until either explicitly retracted via retract! or the immediately enclosing actor exits. level specifies which dataspace the assertion should be made, in terms of relative distance from the dataspace containing the enclosing actor.

procedure

(retract! v [#:meta-level level])  void?

  v : any/c
  level : natural-number/c = 0
Retracts any assertions made by the immediately enclosing actor at level dataspaces above the enclosing dataspace of the form v.

syntax

(state maybe-init (maybe-bindings O ...) ([E I ...] ...))

 
maybe-init = 
  | #:init [I ...]
     
maybe-bindings = 
  | #:collect ([id init] ...)
 
  id : identifier?
Spawns a new actor with ongoing behaviors O ... that runs until a termination event is detected.

The optional #:init [I ...] provides a sequence of initialization actions. The initial actions are executed before the ongoing behaviors begin but after the interests of the state actor are established.

The optional #:collect [(id init) ...] clause introduces bindings that are visible within the body of the state actor. Each binding id is initialized to the corresponding init expression. The bindings are updated when an ongoing behavior executes an instantaneous event, such as the result of an on behavior. The new bindings are in the form of a values form, with the new values in the same order and number as in the #:collect.

The ongoing behaviors O ... are run simultaneously until the state actor exits.

Each [E I ...] specifies a termination event E of the actor. When a termination event E activates, the corresponding Is are executed. The state actor then exits, with the same result of the final I action.

syntax

(until E
       maybe-init
       maybe-bindings
       maybe-done
       O ...)
 
maybe-init = 
  | #:init [I ...]
     
maybe-bindings = 
  | #:collect ([id init] ...)
     
maybe-done = 
  | #:done [I ...]
 
  id : identifier?
An until behavior corresponds to a state behavior with only one termination event, given by E. The final result of the until behavior is the values of the #:collect bindings in scope from any parent actors followed by the final values of the until actor’s bindings. The actions in a #:done clause are executed after the termination event but before the until actor exits.

syntax

(forever maybe-init
  maybe-bindings
  O ...)
 
maybe-init = 
  | #:init [I ...]
     
maybe-bindings = 
  | #:collect ([id init] ...)
 
  id : identifier?
The forever behavior is analogous to a state form with no termination events.