9 When Should I Use Qi?
Okay, so you’ve read Using Qi and understand at a high level that there are interface macros providing a bridge between Racket and Qi, which allow you to use Qi anywhere in your code, and you have some idea of how to write flows (maybe you’ve gone through the Tutorial). Let’s now look at a collection of examples that may help shed light on when you should use Qi vs Racket or another language.
9.1 Hadouken!
When you’re interested in transforming values, Qi is often the right language to use.
9.1.1 Super Smush Numbers
Let’s say we’d like to define a function that adds together all of the input numbers, except that instead of using usual addition, we want to just adjoin them ("smush them") together to create a bigger number as the result.
In Qi, we could express it this way:
(define-flow smush (~> (>< ->string) string-append ->number))
The equivalent in Racket would be:
(define (smush . vs) (->number (apply string-append (map ->string vs))))
The Qi version uses >< to "map" all input values under the ->string flow to convert them to strings. Then it appends these values together as strings, finally converting the result back to a number to produce the result.
The Racket version needs to be parsed in detail in order to be understood, while the Qi version reads sequentially in the natural order of the transformations, and makes plain what these transformations are. Qi is the natural choice here.
The documentation for the threading macro contains additional examples of such transformations which to a first approximation apply to Qi as well (see Relationship to the Threading Macro).
9.1.2 Root-Mean-Square
While you can always use Qi to express computations as flows, it isn’t always a better way of thinking about them – just a different way, more appropriate in some cases and less in others. Let’s look at an example that illustrates this subjectivity.
The "root mean square" is a measure often used in statistics to estimate the magnitude of some quantity for which there are many independent measurements. For instance, given a set of values representing the "deviation" of the result from the predictions of a model, we can use the square root of the mean of the squares of these values as an estimate of "error" in the model, i.e. inversely, an estimate of the accuracy of the model. The RMS also comes up in other branches of engineering and mathematics. What if we encountered such a case and wanted to implement this function? In Racket, we might implement it as:
(define (rms vs) (sqrt (/ (apply + (map sqr vs)) (length vs))))
In Qi, it would be something like:
(define-flow rms (~> △ (-< (~> (>< sqr) +) count) / sqrt))
This first uses the "prism" △ to separate the input list into its component values. Then it uses a tee junction, -<, to fork these values down two flows, one to compute the sum of squares and the other to count how many values there are. In computing the sum of squares, >< "maps" the input values under the sqr flow to yield the squares of the input values which are then summed. This is then divided by the count to yield the mean of the squares, whose square root is then produced as the result.
Here, there are reasons to favor either representation. The Racket version doesn’t have too much redundancy so it is a fine way to express the computation. The Qi version eliminates the redundant references to the input (as it usually does), but aside from that it is primarily distinguished as being a way to express the computation as a series of transformations evaluated sequentially, while the Racket version expresses it as a compound expression to be evaluated hierarchically. They’re just different and neither is necessarily better.
9.2 The Science of Deduction
When you seek to analyze some values and make inferences or assertions about them, or take certain actions based on observed properties of values, or, more generally, when you want to express anything exhibiting subject-predicate structure, Qi is often the right language to use.
9.2.1 Compound Predicates
In Racket, if we seek to make a compound assertion about some value, we might do something like this:
(λ (num) (and (positive? num) (integer? num) (= 0 (remainder num 3))))
This recognizes positive integers divisible by three. Using the utilities in Additional Higher-Order Functions, we might write it as:
(conjoin positive? integer? (compose (curry = 0) (curryr remainder 3)))
... which avoids the wrapping lambda, doesn’t mention the argument redundantly, and transparently encodes the fact that the function is a compound predicate. On the other hand, it is arguably less easy on the eyes. For starters, it uses the word "conjoin" to avoid colliding with "and," to refer to a similar idea. It also uses the words "curry" and "curryr" to partially apply functions, which are somewhat gratuitous as ways of saying "equal to zero" and "remainder by three."
In Qi, this would be written as:
(and positive? integer? (~> (remainder 3) (= 0)))
They say that perfection is achieved not when there is nothing left to add, but when there is nothing left to take away. Well then.
9.2.2 abs
Let’s say we want to implement abs. This is a function that returns the absolute value of the input argument, i.e. the value unchanged if it is positive, and negated otherwise – a conditional transformation. With Racket, we might implement it like this:
(define (abs v) (if (negative? v) (- v) v))
For this very simple function, the input argument is mentioned four times! An equivalent Qi definition is:
(define-switch abs [negative? -] [else _])
This uses the definition form of switch, which is a flow-oriented conditional analogous to cond. The _ symbol here indicates that the input is to be passed through unchanged, i.e. it is the trivial or identity flow. The input argument is not mentioned; rather, the definition expresses abs as a conditioned transformation of the input, that is, the essence of what this function is.
Technically, as the switch form transforms the input based on conditions, if none of the conditions match, no transformation is applied, and the inputs are produced, unchanged. So abs could also be implemented simply as:
(define-switch abs [negative? -])
9.2.3 Range
[This example was suggested by user Rubix on the Racket Discord]
We’d like to find the greatest difference in a set of values, which in statistical applications is called the range. This is how we’d do it in Qi:
(define-flow range (~> △ (-< max min) -))
This separates the input list into its component values and passes those values independently through the functions max and min before taking the difference between them.
The code in Racket would be:
(define (range xs) (- (apply max xs) (apply min xs)))
The Racket version mentions the input three times and needs to "lift" the max and min functions so that they are applicable to lists rather than values. The Qi version is about as economical an implementation as you will find, expressing the essential idea and nothing more.
9.2.4 Length
Calculating the length of a list is a straightforward computation. Here are a few different ways to do it in Racket:
(define (length lst) (if (empty? lst) 0 (add1 (length (rest lst))))) (define (length lst) (apply + (map (const 1) lst))) (define length (compose (curry apply +) (curry map (const 1)))) (define (length vs) (foldl (λ (v acc) (add1 acc)) 0 vs)) (define (length vs) (for/sum ([_ vs]) 1))
And here it is in Qi:
(define-flow length (~> △ (>< 1) +))
This separates the input list into its component values, produces a 1 corresponding to each value, and then adds these ones together to get the length. It is the same idea encoded (and indeed, hidden) in some of the Racket implementations.
This succinctness is possible because Qi reaps the twin benefits of (1) working directly with values (and not just collections of values), and (2) Racket’s support for variadic functions that accept any number of inputs (in this case, +).
9.3 Don’t Stop Me Now
When you’re interested in functionally transforming lists using operations like map, filter, foldl and foldr, Qi is a good choice because its optimizing compiler eliminates intermediate representations that would ordinarily be constructed in computing the result of such a sequence, resulting in significant performance gains in some cases.
For example, consider the Racket function:
(define (filter-map vs) (map sqr (filter odd? vs)))
In evaluating this sequence, the input list is traversed to produce the result of filter, which is a list that is traversed one more time to produce another list that is the result of map.
The equivalent Qi flow is:
(define-flow filter-map (~> (filter odd?) (map sqr)))
Here, under the hood, each element of the input list is processed one at a time, with both of these functions being invoked on it in sequence, and then the output list is constructed by accumulating these individual results. This ensures that no intermediate lists are constructed along the way and that the input list is traversed just once – a standard optimization technique called "stream fusion" or "deforestation." The Qi version produces the same result as the Racket code above, but it can be both faster as well as more memory-efficient, especially on large input lists. Note however that if the functions used in filter and map are not pure, that is, if they perform side effects like printing to the screen or writing to a file, then the Qi flow would exhibit a different order of effects than the Racket version.
9.4 Curbing Curries and Losing Lambdas
Since flows are just functions, you can use them anywhere that you would normally use a function. In particular, they are often a clearer alternative to using currying or lambdas. For instance, to double every number in a list, we could do:
(map (☯ (* 2)) (range 10))
... rather than use currying:
(map (curry * 2) (range 10))
... or a lambda:
(map (λ (v) (* v 2)) (range 10))
The Qi version expresses the essential idea without introducing arcane concepts (such as currying).
9.5 The Value in Values
Racket exhibits a satisfying symmetry between input arguments and return values, allowing functions to return multiple values just as they accept multiple arguments. But syntactically, working with multiple values in Racket can be cumbersome, as we see in this example where we simply collect the return values of the built-in time-apply utility into a list:
(call-with-values (λ _ (time-apply + (list 1 2 3))) list)
The symmetry between arguments and return values is even more apparent and natural in Qi, where functions are seen as flows, and arguments and return values as inputs and outputs, respectively. Thus, a function returning multiple values is just another flow and doesn’t require special handling, making Qi a good choice in such cases:
(~> () (time-apply + (list 1 2 3)) list)
9.6 Making the Switch
Scheme code in the wild is littered with cond expressions resembling these:
(cond [(positive? v) (add1 v)] [(negative? v) (sub1 v)] [else v])
(cond [(>= a b) (- a b)] [(< a b) (- b a)])
... or even if expressions like these:
(if (>= a b) (- a b) (- b a))
Such expressions are conditional transformations of the inputs, but this idea is nowhere encoded in the expressions themselves, leading to repetition, duplication, recapitulation, and even redundancy. In such cases, switch to switch:
(switch (v) [positive? add1] [negative? sub1])
(switch (a b) [>= -] [< (~> X -)])
switch is a versatile conditional form that can also express more complex flows, as we will see in the next example.
9.7 The Structure and Interpretation of Flows
Sometimes, it is natural to express the entire computation as a flow, while at other times it may be better to express just a part of it as a flow. In either case, the most natural representation may not be apparent at the outset, by virtue of the fact that we don’t always understand the computation at the outset. In such cases, it may make sense to take an incremental approach.
The classic Computer Science textbook, "The Structure and Interpretation of Computer Programs," contains the famous "metacircular evaluator" – a Scheme interpreter written in Scheme. The code given for the eval function is:
(define (eval exp env) (cond [(self-evaluating? exp) exp] [(variable? exp) (lookup-variable-value exp env)] [(quoted? exp) (text-of-quotation exp)] [(assignment? exp) (eval-assignment exp env)] [(definition? exp) (eval-definition exp env)] [(if? exp) (eval-if exp env)] [(lambda? exp) (make-procedure (lambda-parameters exp) (lambda-body exp) env)] [(begin? exp) (eval-sequence (begin-actions exp) env)] [(cond? exp) (eval (cond->if exp) env)] [(application? exp) (apply (eval (operator exp) env) (list-of-values (operands exp) env))] [else (error "Unknown expression type -- EVAL" exp)]))
This implementation in Racket mentions the expression to be evaluated, exp, twenty-five times. This kind of redundancy is often a sign that the computation can be profitably thought of as a flow. In the present case, we notice that every condition in the cond expression is a predicate applied to exp. It would seem that it is the expression exp that flows, here, through a series of checks and transformations in the context of some environment env. By modeling the computation this way, we derive the following implementation:
(define (eval exp env) (switch (exp) [self-evaluating? _] [variable? (lookup-variable-value env)] [quoted? text-of-quotation] [assignment? (eval-assignment env)] [definition? (eval-definition env)] [if? (eval-if env)] [lambda? (~> (-< lambda-parameters lambda-body) (make-procedure env))] [begin? (~> begin-actions (eval-sequence env))] [cond? (~> cond->if (eval env))] [application? (~> (-< (~> operator (eval env)) (~> operands △ (>< (eval env)))) apply)] [else (error "Unknown expression type -- EVAL" _)]))
This version eliminates two dozen redundant references to the input expression that were present in the original Racket implementation, and reads naturally. As it uses partial application templates in the consequent flows, this version could be considered a hybrid implementation in Qi and Racket.
Yet, an astute observer may point out that although this eliminates almost all mention of exp, that it still contains ten references to the environment, env. In our first attempt at a flow-oriented implementation, we chose to see the eval function as a flow of the input expression through various checks and transformations. We were led to this choice by the observation that all of the conditions in the original Racket implementation were predicated exclusively on exp. But now we see that almost all of the consequent expressions use the environment, in addition. That is, it would appear that the environment env flows through the consequent expressions.
For such cases, by means of a divert (or its alias, %) clause "at the floodgates," the switch form allows us to control which values flow to the predicates and which ones flow to the consequents. In the present case, we’d like the predicates to only receive the input expression, and the consequents to receive both the expression as well as the environment. By modeling the flow this way, we arrive at the following pure-Qi implementation.
(define-switch eval (% 1> _) [self-evaluating? 1>] [variable? lookup-variable-value] [quoted? (~> 1> text-of-quotation)] [assignment? eval-assignment] [definition? eval-definition] [if? eval-if] [lambda? (~> (== (-< lambda-parameters lambda-body) _) make-procedure)] [begin? (~> (== begin-actions _) eval-sequence)] [cond? (~> (== cond->if _) eval)] [application? (~> (-< (~> (== operator _) eval) (~> (== operands _) (△ eval))) apply)] [else (error "Unknown expression type -- EVAL" 1>)])
This version eliminates the more than thirty mentions of the inputs to the function that were present in the Racket version, while introducing four flow references (i.e. 1>). Some of the clauses are unsettlingly elementary, reading like pseudocode rather than a real implementation, while other clauses become complex flows reflecting the path the inputs take through the expression. This version is stripped down to the essence of what the eval function does, encoding a lot of our understanding syntactically that otherwise is gleaned only by manual perusal – for instance, the fact that all of the predicates are only concerned with the input expression is apparent on the very first line of the switch body. The complexity in this implementation reflects the complexity of the computation being modeled, nothing more.
While the purist may favor this last implementation, it is perhaps a matter of some subjectivity. We were led to Qi in this instance by the evidence of redundancy in the implementation, which we took to be a clue that this could be modeled as a flow. It wasn’t obvious at the outset that this was the case. Some may see this as evidence that a flow isn’t the "natural" way to think about this computation. Others may disagree with this position, citing that it’s difficult for the intuition to always penetrate the fog of complexity, and employing evidence to reinforce our intuitions is precisely how we can see farther, and that, as the evidence in this case suggested it was a flow, that it is, in fact, best thought of as a flow. Wherever you may find your sympathies to lie on this spectrum, objectively, we find that the pure-Qi solution is the most economical both conceptually as well as lexically (i.e. the shortest in terms of number of characters), while the hybrid solution is just a little more verbose. The original Racket implementation is in third place on both counts.
9.8 Using the Right Tool for the Job
We’ve seen a number of examples covering transformations, predicates, and conditionals, both simple and complex, where using Qi to describe the computation was often a natural and elegant choice, though not always an obvious one.
The examples hopefully illustrate an age-old doctrine – use the right tool for the job. A language is the best tool of all, so use the right language to express the task at hand. Sometimes, that language is Qi and sometimes it’s Racket and sometimes it’s a combination of the two, or something else. Don’t try too hard to coerce the computation into one way of looking at things. It’s less important to be consistent and more important to be fluent and clear. And by the same token, it’s less important for you to fit your brain to the language and more important for the language to be apt to describe the computation, and consequently for it to encourage a way of thinking about the problem that fits your brain.
Employing a potpourri of general purpose and specialized languages, perhaps, is the best way to flow!