On this page:
8.1 What is a Flow?
8.2 Values and Flows
8.3 Flows as Graphs
8.4 Values are Not Collections
8.5 Counting Flows
8.6 Flowy Logic
8.7 Phrases
8.8 Identities
8.9 Flows and Arrows
8.10 It’s Languages All the Way Down

8 Principles of Qi🔗ℹ

After many patient hours meticulously crafting Qi flows, you may find that you seek a deeper understanding; insight into guiding principles and inner workings, so that you can hone your skills on firmer ground.

Welcome. Your wanderings have brought you to the right place. In this section, we will cover various topics that will help you have a fuller understanding and a sound conceptual model of how Qi works. This kind of facility with the fundamentals will be useful as you employ Qi for more complex tasks, enabling you to engage in higher level reasoning about the task at hand rather than be mired in conceptual building blocks.

    8.1 What is a Flow?

    8.2 Values and Flows

    8.3 Flows as Graphs

    8.4 Values are Not Collections

    8.5 Counting Flows

    8.6 Flowy Logic

    8.7 Phrases

    8.8 Identities

    8.9 Flows and Arrows

    8.10 It’s Languages All the Way Down

8.1 What is a Flow?🔗ℹ

A flow is either made up of flows, or is a native (e.g. Racket) function. Flows may be composed using a number of combinators that could yield either linear or nonlinear composite flows.

A flow in general accepts m inputs and yields n outputs, for arbitrary non-negative integers m and n. We say that such a flow is m × n.

The semantics of a flow is function invocation – simply invoke a flow with inputs (i.e. ordinary arguments) to obtain the outputs.

The Qi language allows you to describe and use flows in your code.

8.2 Values and Flows🔗ℹ

Flows accept inputs and produce outputs – they are functions. The things that flow – the inputs and outputs – are values. Yet, values do not actually "move" through a flow, since a flow does not mutate them. The flow simply produces new values that are related to the inputs by a computation.

Every flow is made up of components that are themselves flows. Thus, each of these components is a relationship between an input set of values and an output set of values, so that at every level, flows produce sequences of sets of values beginning with the inputs and ending with the outputs, with each set related to the preceding one by a computation, and again, no real "motion" of values at all.

So indeed, when we say that values "flow," there is nothing in fact that truly flows, and it is merely a convenient metaphor.

8.3 Flows as Graphs🔗ℹ

A flow could also be considered an acyclic graph, with its component flows as nodes, and a directed edge connecting two flows if an output of one is used as an input of the other. There may be many distinct paths that could be traced over this graph, and we may imagine values to flow along these paths at runtime (although of course, there is nothing that flows). At each point in the flow (in this spatial sense), there are a certain number of values present, depending on the runtime inputs. We refer to this number as the arity or the volume of the flow at that point. Volume is a runtime concept since it depends on the actual inputs provided to the flow, although there may be cases where it could be determined at compile time.

8.4 Values are Not Collections🔗ℹ

The things that flow are values. Individual values may happen to be collections such as lists, but the values that are flowing are not, together, a collection of any kind.

To understand this with an example: when we employ a tee junction in a flow, colloquially, we might say that the junction "divides the flow into two," which might suggest that there are now two flows. But in fact, there is just one flow that divides values down two separate flows which are part of its makeup. More precisely, -< composes two flows to yield a single composite flow. Like any flow, this composite flow accepts values and produces values, not collections of values. There is no way to differentiate, at the output end, which values came from the first channel of the junction and which ones came from the second, since downstream flows have no idea about the structure of upstream flows and only see the values they receive.

The way to group values, if we need grouping, is to collect them into a data structure (e.g. a list) using a collection prism, . In the case of a tee junction, the way to differentiate between values coming from each channel of the junction is for the channels to individually collect their values at the end. That way, the values that are the output of the composite flow are lists generated individually by the various channels of the flow.

8.5 Counting Flows🔗ℹ

Everything in Qi is a function. Programs are functions, they are made up of functions. Even literals are interpreted as functions generating them.

Consider this example:

(~> sqr (-< add1 5) *)

There are six flows here, in all: the entire one, each component of the thread, and each component of the tee junction.

8.6 Flowy Logic🔗ℹ

Qi’s design is inspired by buddhist śūnyatā logic. To understand it holistically would require a history lesson to put the sunyata development in context, and that would be quite a digression. But in essence, sunyata is about transcension of context or viewpoint. A viewpoint is identifiable with a logical span of possibilities (catuṣkoṭi) in terms of which assertions may be made. Sunyata is the rejection of all of the available logical possibilities, thus transcending the very framing of the problem (this is signified by the word mu in Zen). This kind of transcension could suggest alternative points of view, but more precisely, does not indicate a point of view (which isn’t the same as being ambivalent or even agnostic). This idea has implications not just for formal logical systems but also for everyday experience and profound metaphysical questions alike.

But for the purposes of Qi, what it means is that the existence of a value is a logical span within which it takes on specific forms. Sunyata is the difference between a value taking on a form indicating tangible output (e.g. 5 or "hello") or indicating absence (e.g. (void) or "") or failure (e.g. #f), or provisionality (e.g. 'suspended) or certainty (e.g. #t) – it’s the difference between these, and not existing at all.

The same considerations extend the other way as well, from nonexistence to existence to existence of more than one. As each value corresponds to an independent logical span of possibilities, sunyata in the context of Qi translates into the core paradigm being the existence/non-existence and consequently also the number of values at each point in the flow.

In practice, this means that Qi will often opt to either return or not return a value rather than return a value signifying absence or raise an error. This principle even suggests considerations for the design of ordinary functions and the evaluator itself, which from a Qi/sunyata perspective, could model absence and number in positions that typically expect values.

For example, the following would seem to be in accord with these principles:

See Write Yourself a Maybe Monad for Great Good for an example that applies some of these ideas to implement the Maybe monad commonly used in many functional languages. Although, note that if the above design were adopted by the underlying language and interpreter, Maybe would be unnecessary in most cases.

8.7 Phrases🔗ℹ

When reading languages like English, we understand what we read in terms of words and then phrases and the relationship of these phrases to one another as clauses of a sentence, and then the relationship of sentences to one another, and then paragraphs to one another. The resourceful speed readers among us even do this in the reverse order at first, discerning high level structure before parsing the low level component meanings. Just like human languages, Qi expressions exhibit phrase structure that we can leverage in similar ways. Here are some common phrases in the language to get you started thinking about the language in this way.

Some of these phrases may someday make it into the language as forms themselves, and there may be higher-level phrases still, made up of such phrases.

8.8 Identities🔗ℹ

Here are some useful identities for the core routing forms. They can be used to simplify your code or say things in different ways.

(\sim> (\sim> f g) h) = (\sim> f (\sim> g h)) = (\sim> f g h)
[associative law]
(\sim> f \_) = (\sim> \_ f) = (\sim> f) = f
[left and right identity]
(== (\sim> f₁ g₁) (\sim> f₂ g₂)) = (\sim> (== f₁ f₂) (== g₁ g₂))
(\sim> (>< f) (>< g)) = (>< (\sim> f g))
(\sim> \_ \cdots) = \_
(== \_ \ldots) = \_
(>< \_) = \_
(-< f) = f
(-< (\text{gen} a) (\text{gen} b)) = (-< (\text{gen} a b))
(\sim> △ ▽) = \_ = (\sim> ▽ △) \text{(the former only holds when the input is a list)}

8.9 Flows and Arrows🔗ℹ

[The connection between flows and arrows was pointed out by Sergiu Ivanov (Scolobb on Discourse).]

It turns out that the core routing forms of Qi fulfill the definition of arrows in category theory and Haskell (in an unfortunate conflation of terminology, these "arrows" are an entirely different notion than morphisms, which are also sometimes referred to as arrows). The specific correspondence is as follows:

So evidently, flows are just monoids in suitable subcategories of bifunctors (what’s the problem?), or, in another way of looking at it, enriched Freyd categories.

Therefore, any theoretical results about arrows should generally apply to Qi as well (but not necessarily, since Qi is not just arrows).

8.10 It’s Languages All the Way Down🔗ℹ

Qi is a language implemented on top of another language, Racket, by means of a macro called flow. All of the other macros that serve as Qi’s embedding into Racket, such as (the Racket macros) ~> and switch, expand to a use of flow.

The flow form accepts Qi syntax and (like any macro) produces Racket syntax. It does this in two stages:

  1. Expansion, where the Qi source expression is translated to a small core language (Core Qi).

  2. Compilation, where the Core Qi expression is optimized and then translated into Racket.

All of this happens at compile time, and consequently, the generated Racket code is then itself expanded to a small core language and then compiled to bytecode for evaluation in the runtime environment, as usual.

Thus, Qi is a special kind of hosted language, one that happens to have the same architecture as the host language, Racket, in terms of having distinct expansion and compilation steps. This gives it a lot of flexibility in its implementation, including allowing much of its surface syntax to be implemented as Qi macros (for instance, Qi’s switch expands to a use of Qi’s if just as Racket’s cond expands to a use of Racket’s if), allowing it to be naturally macro-extensible by users, and lending it the ability to perform optimizations on the core language that allow idiomatic code to be performant.

This architecture is achieved through the use of Syntax Spec, following the general approach described in Macros for Domain-Specific Languages (Ballantyne et. al.).