1.4 Specifying Syntax with Syntax Classes

Syntax classes provide an abstraction mechanism for syntax patterns. Built-in syntax classes are supplied that recognize basic classes such as identifier and keyword. Programmers can compose basic syntax classes to build specifications of more complex syntax, such as lists of distinct identifiers and formal arguments with keywords. Macros that manipulate the same syntactic structures can share syntax class definitions.

A syntax class may have formal parameters, in which case they are bound as variables in the body. Syntax classes support optional arguments and keyword arguments using the same syntax as lambda. The body of the syntax-class definition contains a non-empty sequence of pattern variants.

The following options are supported:

#:attributes (attr-arity-decl ...)

attr-arity-decl   =   attr-id     |   (attr-id depth)

Declares the attributes of the syntax class. An attribute arity declaration consists of the attribute name and optionally its ellipsis depth (zero if not explicitly specified).

If the attributes are not explicitly listed, they are inferred as the set of all pattern variables occurring in every variant of the syntax class. Pattern variables that occur at different ellipsis depths are not included, nor are nested attributes from annotated pattern variables.

#:auto-nested-attributes

Deprecated. This option cannot be combined with #:attributes.

Declares the attributes of the syntax class as the set of all pattern variables and nested attributes from annotated pattern variables occurring in every variant of the syntax class. Only syntax classes defined strictly before the enclosing syntax class are used to compute the nested attributes; pattern variables annotated with not-yet-defined syntax classes contribute no nested attributes for export. Note that with this option, reordering syntax-class definitions may change the attributes they export.

#:description description-expr

description-expr   :   (or/c string? #f)

The description argument is evaluated in a scope containing the syntax class’s parameters. If the result is a string, it is used in error messages involving the syntax class. For example, if a term is rejected by the syntax class, an error of the form "expected description" may be synthesized. If the result is #f, the syntax class is skipped in the search for a description to report.

If the option is not given, the name of the syntax class is used instead.

#:opaque

Indicates that errors should not be reported with respect to the internal structure of the syntax class.

#:commit

Directs the syntax class to “commit” to the first successful match. When a variant succeeds, all choice points within the syntax class are discarded. See also ~commit.

#:no-delimit-cut

By default, a cut (~!) within a syntax class only discards choice points within the syntax class. That is, the body of the syntax class acts as though it is wrapped in a ~delimit-cut form. If #:no-delimit-cut is specified, a cut may affect choice points of the syntax class’s calling context (another syntax class’s patterns or a syntax-parse form).

It is an error to use both #:commit and #:no-delimit-cut.

Each variant of a syntax class is specified as a separate pattern-form whose syntax pattern is a single-term pattern.

The options are the same as for define-syntax-class.

Each variant of a splicing syntax class is specified as a separate pattern-form whose syntax pattern is a head pattern.

When used within define-syntax-class, syntax-pattern should be a single-term pattern; within define-splicing-syntax-class, it should be a head pattern.

The attributes of the variant are the attributes of the pattern together with all attributes bound by #:with clauses, including nested attributes produced by syntax classes associated with the pattern variables.

Raises an error when used as an expression outside of a syntax-class definition or syntax-parse expression.

When a transformer is bound to an instance of a struct with this property, then it may be used as a syntax class or splicing syntax class in the same way as the bindings created by define-syntax-class or define-splicing-syntax-class. If the value of the property is an identifier, then it should be bound to a syntax class or splicing syntax class, and the binding will be treated as an alias for the referenced syntax class. If the value of the property is a procedure, then it will be applied to the value with the prop:syntax-class property to obtain an identifier, which will then be used as in the former case.

Added in version 7.2.0.4 of package base.

1.4.1 Pattern Directives

Both the parsing forms and syntax class definition forms support pattern directives for annotating syntax patterns and specifying side conditions. The grammar for pattern directives follows:

#:declare pvar-id stxclass maybe-role

stxclass   =   syntax-class-id     |   (syntax-class-id arg ...)           maybe-role   =       |   #:role role-expr

Associates pvar-id with a syntax class and possibly a role, equivalent to replacing each occurrence of pvar-id in the pattern with (~var pvar-id stxclass maybe-role). The second form of stxclass allows the use of parameterized syntax classes, which cannot be expressed using the “colon” notation. The args are evaluated in the scope where the pvar-id occurs in the pattern. Keyword arguments are supported, using the same syntax as in #%app.

If a #:with directive appears between the main pattern (e.g., in a syntax-parse or define-syntax-class clause) and a #:declare, then only pattern variables from the #:with pattern may be declared.

Examples:

> (syntax-parse #'P     [x      #:declare x id      #'x])

#<syntax:eval:12:0 P>

> (syntax-parse #'L     [x      #:with y #'x      #:declare x id      #'x])

syntax-parse: identifier in #:declare clause does not appear

in pattern;

this #:declare clause affects only the preceding #:with

pattern

at: x

in: (syntax-parse (syntax L) (x #:with y (syntax x)

#:declare x id (syntax x)))

> (syntax-parse #'T     [x      #:with y #'x      #:declare y id      #'x])

#<syntax:eval:14:0 T>

#:post action-pattern

Executes the given action pattern as a “post-traversal check” after matching the main pattern. That is, the following are equivalent:

main-pattern #:post action-pattern

main-pattern #:and (~post action-pattern)

(~and main-pattern (~post action-pattern))

#:and action-pattern

Like #:post except that no ~post wrapper is added. That is, the following are equivalent:

main-pattern #:and action-pattern

(~and main-pattern action-pattern)

#:with syntax-pattern stx-expr

Evaluates the stx-expr in the context of all previous attribute bindings and matches it against the pattern. If the match succeeds, the pattern’s attributes are added to environment for the evaluation of subsequent side conditions. If the #:with match fails, the matching process backtracks. Since a syntax object may match a pattern in several ways, backtracking may cause the same clause to be tried multiple times before the next clause is reached.

If the value of stx-expr is not a syntax object, it is implicitly converted to a syntax object. If the the conversion would produce 3D syntax—that is, syntax that contains unwritable values such as procedures, non-prefab structures, etc—then an exception is raised instead.

Equivalent to #:post (~parse syntax-pattern stx-expr).

Examples:

> (syntax-parse #'(1 2 3)     [(a b c)      #:with rev #'(c b a)      #'rev])

#<syntax:eval:15:0 (3 2 1)>

> (syntax-parse #'(['x "Ex."] ['y "Why?"] ['z "Zee!"])     [([stuff ...] ...)      #:with h #'(hash stuff ... ...)      #'h])

#<syntax:eval:16:0 (hash (quote x) "Ex." (quote y) "Why?" (quote z) "Zee!")>

#:attr attr-arity-decl expr

Evaluates the expr in the context of all previous attribute bindings and binds it to the given attribute. The value of expr need not be, or even contain, syntax—see attribute for details.

Equivalent to #:and (~bind attr-arity-decl expr).

Examples:

> (syntax-parse #'("do" "mi")     [(a b)      #:attr rev #'(b a)      #'rev])

#<syntax:eval:17:0 ("mi" "do")>

> (syntax-parse #'(1 2)     [(a:number b:number)      #:attr sum (+ (syntax-e #'a) (syntax-e #'b))      (attribute sum)])

3

The #:attr directive is often used in syntax classes:

Examples:

> (define-syntax-class ab-sum     (pattern (a:number b:number)       #:attr sum (+ (syntax-e #'a) (syntax-e #'b))))

> (syntax-parse #'(1 2)     [x:ab-sum      (attribute x.sum)])

3

#:fail-when condition-expr message-expr

message-expr   :   (or/c string? #f)

Evaluates the condition-expr in the context of all previous attribute bindings. If the value is any true value (not #f), the matching process backtracks (with the given message); otherwise, it continues. If the value of the condition expression is a syntax object, it is indicated as the cause of the error.

If the message-expr produces a string it is used as the failure message; otherwise the failure is reported in terms of the enclosing descriptions.

Equivalent to #:post (~fail #:when condition-expr message-expr).

Examples:

> (syntax-parse #'(m 4)     [(m x:number)      #:fail-when (even? (syntax-e #'x))      "expected an odd number"      #'x])

m: expected an odd number

at: (m 4)

in: (m 4)

> (syntax-parse #'(m 4)     [(m x:number)      #:fail-when (and (even? (syntax-e #'x)) #'x)      "expected an odd number"      #'x])

m: expected an odd number

at: 4

in: (m 4)

#:fail-unless condition-expr message-expr

message-expr   :   (or/c string? #f)

Like #:fail-when with the condition negated.

Equivalent to #:post (~fail #:unless condition-expr message-expr).

Example:

> (syntax-parse #'(m 5)     [(m x:number)      #:fail-unless (even? (syntax-e #'x))      "expected an even number"      #'x])

m: expected an even number

at: (m 5)

in: (m 5)

#:when condition-expr

Evaluates the condition-expr in the context of all previous attribute bindings. If the value is #f, the matching process backtracks. In other words, #:when is like #:fail-unless without the message argument.

Equivalent to #:post (~fail #:unless condition-expr #f).

Example:

> (syntax-parse #'(m 5)     [(m x:number)      #:when (even? (syntax-e #'x))      #'x])

m: bad syntax

in: (m 5)

#:do [defn-or-expr ...]

Takes a sequence of definitions and expressions, which may be intermixed, and evaluates them in the scope of all previous attribute bindings. The names bound by the definitions are in scope in the expressions of subsequent patterns and clauses.

There is currently no way to bind attributes using a #:do block. It is an error to shadow an attribute binding with a definition in a #:do block.

Equivalent to #:and (~do defn-or-expr ...).

#:undo [defn-or-expr ...]

Has no effect when initially matched, but if backtracking returns to a point before the #:undo directive, the defn-or-exprs are executed. See ~undo for an example.

Equivalent to #:and (~undo defn-or-expr ...).

#:cut

Eliminates backtracking choice points and commits parsing to the current branch at the current point.

Equivalent to #:and ~!.

1.4.2 Pattern Variables and Attributes

An attribute is a name bound by a syntax pattern. An attribute can be a pattern variable itself, or it can be a nested attribute bound by an annotated pattern variable. The name of a nested attribute is computed by concatenating the pattern variable name with the syntax class’s exported attribute’s name, separated by a dot (see the example below).

Attributes can be used in three ways: with the attribute form; inside syntax templates via syntax, quasisyntax, etc; and inside datum templates. Attribute names cannot be used directly as expressions; that is, attributes are not variables.

A syntax-valued attribute is an attribute whose value is a syntax object or list of the appropriate ellipsis depth. That is, an attribute with ellipsis depth 0 is syntax-valued if its value is syntax?; an attribute with ellipis depth 1 is syntax-valued if its value is (listof syntax?); an attribute with ellipsis depth 2 is syntax-valued if its value is (listof (listof syntax?)); and so on. The value is considered syntax-valued if it contains promises that when completely forced produces a suitable syntax object or list. Syntax-valued attributes can be used within syntax, quasisyntax, etc as part of a syntax template. If an attribute is used inside a syntax template but it is not syntax-valued, an error is signaled.

There are uses for non-syntax-valued attributes. A non-syntax-valued attribute can be used to return a parsed representation of a subterm or the results of an analysis on the subterm. A non-syntax-valued attribute must be bound using the #:attr directive or a ~bind pattern; #:with and ~parse will convert the right-hand side to a (possibly 3D) syntax object.

The table syntax class provides four attributes: key, value, hashtable, and sorted-kv. The hashtable attribute has ellipsis depth 0 and the rest have depth 1; key, value, and sorted-kv are syntax-valued, but hashtable is not. The sorted-kv attribute’s value is a promise; it will be automatically forced if used in a template.

Syntax-valued attributes can be used in syntax templates:

> (syntax-parse #'((a 3) (b 2) (c 1))     [t:table      #'(t.key ...)])

#<syntax:eval:26:0 (a b c)>

> (syntax-parse #'((a 3) (b 2) (c 1))     [t:table      #'(t.sorted-kv ...)])

sorting!

#<syntax:eval:27:0 ((c 1) (b 2) (a 3))>

But non-syntax-valued attributes cannot:

> (syntax-parse #'((a 3) (b 2) (c 1))     [t:table      #'t.hashtable])

t.hashtable: attribute contains non-syntax value

value: '#hash((a . 3) (b . 2) (c . 1))

in: t.hashtable

The attribute form gets the value of an attribute, whether it is syntax-valued or not.

> (syntax-parse #'((a 1) (b 2) (c 3))     [t:table      (attribute t.hashtable)])

'#hash((a . 1) (b . 2) (c . 3))

> (syntax-parse #'((a 3) (b 2) (c 1))     [t:table      (attribute t.sorted-kv)])

#<promise:sorted-kv496>

Every attribute has an associated ellipsis depth that determines how it can be used in a syntax template (see the discussion of ellipses in syntax). For a pattern variable, the ellipsis depth is the number of ellipses the pattern variable “occurs under” in the pattern. An attribute bound by #:attr has depth 0 unless declared otherwise. For a nested attribute the depth is the sum of the annotated pattern variable’s depth and the depth of the attribute exported by the syntax class.

Consider the following code:

(define-syntax-class quark

(pattern (a b ...)))

(syntax-parse some-term

[(x (y:quark ...) ... z:quark)

some-code])

The syntax class quark exports two attributes: a at depth 0 and b at depth 1. The syntax-parse pattern has three pattern variables: x at depth 0, y at depth 2, and z at depth 0. Since y and z are annotated with the quark syntax class, the pattern also binds the following nested attributes: y.a at depth 2, y.b at depth 3, z.a at depth 0, and z.b at depth 1.

An attribute’s ellipsis nesting depth is not a guarantee that it is syntax-valued or has any list structure. In particular, ~or* and ~optional patterns may result in attributes with fewer than expected levels of list nesting, and #:attr and ~bind can be used to bind attributes to arbitrary values.

1.4.2.1 Attributes and datum

The datum form is another way, in addition to syntax and attribute, of using syntax pattern variables and attributes. Unlike syntax, datum does not require attributes to be syntax-valued. Wherever the syntax form would create syntax objects based on its template (as opposed to reusing syntax objects bound by pattern variables), the datum form creates plain S-expressions.

Continuing the table example from above, we can use datum with the key attribute as follows:

> (syntax-parse #'((a 1) (b 2) (c 3))     [t:table (datum (t.key ...))])

'(#<syntax:eval:32:0 a> #<syntax:eval:32:0 b> #<syntax:eval:32:0 c>)

A datum template may contain multiple pattern variables combined within some S-expression structure:

> (syntax-parse #'((a 1) (b 2) (c 3))     [t:table (datum ([t.key t.value] ...))])

'((#<syntax:eval:33:0 a> #<syntax:eval:33:0 1>)   (#<syntax:eval:33:0 b> #<syntax:eval:33:0 2>)   (#<syntax:eval:33:0 c> #<syntax:eval:33:0 3>))

A datum template can use the ~@ and ~? template forms:

> (syntax-parse #'((a 1) (b 2) (c 3))     [t:table (datum ((~@ t.key t.value) ...))])

'(#<syntax:eval:34:0 a>   #<syntax:eval:34:0 1>   #<syntax:eval:34:0 b>   #<syntax:eval:34:0 2>   #<syntax:eval:34:0 c>   #<syntax:eval:34:0 3>)

> (syntax-parse #'((a 56) (b 71) (c 13))     [t:table (datum ((~@ . t.sorted-kv) ...))])

sorting!

'(#<syntax:eval:35:0 c>   #<syntax:eval:35:0 13>   #<syntax:eval:35:0 a>   #<syntax:eval:35:0 56>   #<syntax:eval:35:0 b>   #<syntax:eval:35:0 71>)

> (syntax-parse #'( ((a 1) (b 2) (c 3)) ((d 4) (e 5)))     [(t1:table (~or* t2:table #:nothing))      (datum (t1.key ... (~? (~@ t2.key ...))))])

'(#<syntax:eval:36:0 a>   #<syntax:eval:36:0 b>   #<syntax:eval:36:0 c>   #<syntax:eval:36:0 d>   #<syntax:eval:36:0 e>)

> (syntax-parse #'( ((a 1) (b 2) (c 3)) #:nothing)     [(t1:table (~or* t2:table #:nothing))      (datum (t1.key ... (~? (~@ t2.key ...))))])

'(#<syntax:eval:37:0 a> #<syntax:eval:37:0 b> #<syntax:eval:37:0 c>)

However, unlike for syntax, a value of #f only signals a template failure to ~? if a list is needed for ellipsis iteration, as in the previous example; it does not cause a failure when it occurs as a leaf. Contrast the following:

> (syntax-parse #'( ((a 1) (b 2) (c 3)) #:nothing)     [(t1:table (~or* t2:table #:nothing))      #'(~? t2 skipped)])

#<syntax:eval:38:0 skipped>

> (syntax-parse #'( ((a 1) (b 2) (c 3)) #:nothing)     [(t1:table (~or* t2:table #:nothing))      (datum (~? t2 skipped))])

#f

The datum form is also useful for accessing non-syntax-valued attributes. Compared to attribute, datum has the following advantage: The use of ellipses in datum templates provides a visual reminder of the list structure of their results. For example, if the pattern is (t:table ...), then both (attribute t.hashtable) and (datum (t.hashtable ...)) produce a (listof hash?), but the ellipses make it more apparent.

Changed in version 7.8.0.9 of package base: Added support for syntax pattern variables and attributes to datum.