13 Assessment support
ScienceLogo lets an author declare requirements on a written method. The reference assessor checks their structure: required calls and stage order, supplied workflow parts, and record field presence and value requirements. These checks go beyond parsing syntax, but they do not show that the work happened or judge its scientific adequacy. Protocol conformance, bias analysis, uncertainty analysis, and assessment of observed runs are not built-in checks. Their applications may use the method information the reader preserves. Records and interfaces defines the shared declaration rules for data fields and workflow parts.
13.1 Map workflow parts to interface requirements
Form | What the reference assessor checks |
stage-name implement Protocol.part | Maps a declared stage to an interface part |
procedure implement Protocol.part | Maps a procedure to an interface part |
information.period.u implement Protocol.documentation.uncertainty | Identifies the state value addressing a particular requirement |
must do procedure | Requires a call along the written method's established paths |
must stages in order [First Second ...] | Requires at least two distinct stage names in order |
record-name implement Interface | Checks a written record against a named interface |
record-name implement "relative/file.rkt" | Checks a written record against a separate interface file |
Use implement directly, without must. In an interface, must [...] declares requirements. In a workflow, implement identifies the stage, procedure, or value addressing a requirement. It does not call a procedure, change state, or establish scientific conformity.
Mappings can appear at workflow top level, inside procedures, or inside stages. Stage and procedure references may precede their definitions; a value must be available where it is referenced. The reader checks the references and preserves the mapping and its source location for inspection. must do and must stages in order remain top-level declarations. An assessment reports satisfied, violated, or undetermined for a check. Its outcome must be read together with what that check actually tests.
13.2 Map a pendulum stage and its data
The runnable mapping example uses the protocol from Records and interfaces. The stage addresses its measurement part, and the uncertainty description addresses its nested documentation requirement:
Compact
#lang sciencelogo |
|
workflow "Document a pendulum measurement" |
interface Pendulum-protocol [ |
version "1" |
must [measurement documentation [must [uncertainty] notes]] |
additional-measurements |
] |
performer researcher [kind = "human"] |
|
Timing implement Pendulum-protocol.measurement |
|
stage Timing by researcher from state [ |
information = [period = 2 u [declared = [timing = [estimate = 0.1 unit = "seconds"]]]] |
] |
information.period.u implement Pendulum-protocol.documentation.uncertainty |
end stage with state [ |
information = [period = 2 u [declared = [timing = [estimate = 0.1 unit = "seconds"]]]] |
] |
Verbose
#lang sciencelogo |
|
workflow "Document a pendulum measurement" |
interface Pendulum-protocol [ |
version "1" |
must [measurement documentation [must [uncertainty] notes]] |
additional-measurements |
] |
performer researcher [set kind to "human"] |
|
Timing implements Pendulum-protocol.measurement |
|
start stage Timing by researcher from state [ |
set information to [set period to 2 with uncertainty [set declared to [set timing to [set estimate to 0.1 set unit to "seconds"]]]] |
] |
information.period.u implements Pendulum-protocol.documentation.uncertainty |
end stage with state [ |
set information to [set period to 2 with uncertainty [set declared to [set timing to [set estimate to 0.1 set unit to "seconds"]]]] |
] |
This example declares its interface in the same file, so it also runs from an installed package. In a workflow repository, the existing include rule can load a local interface. A library interface uses its existing import alias, such as prot.Pendulum-protocol.measurement. The interface’s version and source revision identify the protocol being referenced.
A mapping to a parent claims its required descendants. A mapping to one child identifies that child and its parent as present; it does not claim siblings. Optional children remain optional. If an optional group is supplied, its own required children apply. A stage or procedure can reference the whole interface. Several workflow parts may address the same requirement, and one part may address several requirements.
Inspect these declarations with:
racket -l sciencelogo/assessor -- examples/protocol-mapping.rkt |
This reports no structural issues: the required mapping declarations are present. It does not establish that a measurement occurred, that its uncertainty is adequate, or that the protocol was scientifically followed. Applications can inspect the mapped stage, state, and performer to assess those questions.
13.3 Require a call
must do requires a procedure call on the written method’s established paths. For example:
#lang sciencelogo |
|
workflow "How will we compare pendulum periods?" |
must do compare-periods |
do prepare-comparison |
to prepare-comparison [do compare-periods] |
to compare-periods [] |
The assessor reports this condition as satisfied because the workflow calls prepare-comparison, which calls compare-periods. A definition alone, a call inside an unused procedure, or a call only under if false or repeat 0 does not satisfy it. A call that depends on an unresolved condition is reported as undetermined. The check resolves literal conditions and counts; named values and procedure inputs remain unknown. It does not run the procedure or establish that the work happened.
13.4 Require an order of stages
must stages in order [Prepare Measure] requires the named top-level stages to appear in that order in the written workflow. At least two distinct titles are required. Other stages may appear between them. The assessor reports a missing stage or wrong order; a stage inside a procedure does not count as a top-level stage for this check.
13.5 Check a record interface
trial implement Timing checks whether the written shape of trial contains the fields of the named record interface. A literal record can satisfy it even if the record was not constructed from Timing. The path form, trial implement "interfaces/timing.rkt", loads a separate record interface file relative to the workflow file. See Records and interfaces for construction, inheritance, defaults, and the limits of this check when a record’s shape is undetermined.
An interface may also state value requirements such as must [swing-count > 10 unit == "seconds"]. For values available in a written record, the structural checker evaluates these expressions as well as field presence. A false requirement is violated; an unresolved requirement is undetermined. Construction fills defaults with =; an equality requirement written with == supplies no default.
Required data fields reject an explicitly supplied unknown unless their declaration defaults to unknown. This is a violated requirement, reported separately from a missing field. A value the checker cannot resolve from the written method remains undetermined. The permission and the default are preserved in the interface declaration; the field remains required. Equality and inequality can resolve comparisons with explicit unknown markers, including inside records and lists. A value the checker cannot establish stays undetermined instead of being treated as an explicit marker.
Concrete interface defaults also declare the field’s outer value kind. For example, period = 2 requires a number; a known text value violates that requirement. period = unknown number explicitly permits an unknown numeric field. The checker distinguishes a wrong kind from an absent or unknown field, and preserves the default or annotation needed to determine the kind. Records and interfaces gives the declaration rules.
An interface can use for each to apply requirements to every item of a written list. The structural record check examines the known items and identifies missing fields by their positions, such as measurements.[2].period.u. Empty lists have no items to check; explicitly unknown required collections and item fields are rejected unless their declarations permit them. Checks needing permitted unknown or unavailable values remain undetermined. Requirements for every item in a list explains the declaration form. The reserved .count property allows requirements such as must [measurements.count >= 3]. The check computes the number of positions even when their values are unknown. If an unknown collection is explicitly permitted, its count is undetermined. It does not ask for a separate count field or mapping.
For workflow mappings, the model retains the expressions and their guards. The reference mapping checker resolves literal expressions; requirements depending on mapped workflow values remain undetermined. Other applications can use the recorded values and mappings to assess them. List requirements retain their local item name, collection expression, guard, and a nested interface model of the body, including source locations.
13.6 Method information available to other tools
The Racket workflow model records the written investigation as procedures, activities, named input and result slots, imports, performers, named data sources and destinations, interface mappings, must do conditions, and typed relations. read-workflow-model reads an investigation file; read-interface-model reads an interface file. Method elements carry source locations. Optional about descriptions retain their owning workflow, stage, or procedure, Markdown text, and source location. They introduce no activities or assessment conditions; see Describe a workflow, stage, or procedure. Explicit state stages retain their entry and exit records; state calls retain input-location mappings and the return location. A call links to its procedure with invokes; named results link to their producing and using activities. A sequence retains command order. Data assignments retain their dotted paths and link to a declared source or destination with reads-from or writes-to; see Data sources and storage. The assessor checks that each activity, named item, and mapping has one containment link to its recorded parent. A mapping retains whether its subject is a stage, procedure, or value, its resolved reference or value path, the interface version and source revision, and the nested requirement path. Interface models also retain the declarations needed to interpret named is tests, including resolved library definitions. An optional field’s field-constraint is scoped to its presence entry and is checked when that field is supplied. It also retains the declaration location and scope. Multiple mappings remain separate records.
assess-workflow checks this model; assess-file reads the file first. The returned structural assessment includes condition checks and findings that identify their targets. For a required workflow part, satisfied means a matching mapping declaration exists. A missing part is violated. A downstream part reached through affects is reported as possibly affected, with an undetermined outcome for that impact. This assessment does not verify the content of a provision or a run.
The older file-reference form, implements "relative/file.rkt" as alias [provide part by "reference"], remains supported for references written as text. The assessor self-description exercises that form and its missing-part reports. Direct implement mappings identify workflow stages, procedures, and values themselves.
The pendulum timing-plan example can be read into the same model. It returns and prints a timing plan; it does not time a pendulum. The young Galileo case explains how a research question develops through observations and conclusions. Its prose account is distinct from the runnable timing plan and from evidence that measurements occurred. The shared representation contract describes what readers in other languages must preserve. Independent applications use that method information for scientific assessment; the Racket assessor provides the structural checks described here.