The yodl program is a single-pass interpreter of its input files. The program does not build an internal data representation of the text and commands that it encounters; therefore, all actions must be completed in one pass. The basic way by which yodl does this, is by reading input, taking actions and possibly `pushing back' information on the input. This is best illustrated with an example. Given the code,

DEFINESYMBOL(sym)

IFDEF(sym)
    (Symbol sym is defined!
     DEFINEMACRO(testmac)(0)(Testmac now stands for one thing.)
    )
    (Symbol sym is not defined!
     DEFINEMACRO(testmac)(0)(Testmac now stands for another thing.)
    )
    
testmac()

yodl will take the following actions:

The DEFINESYMBOL command updates an internal table of symbols, no other action is necessary here.

When the IFDEF command is parsed, yodl eats up all of the necessary parts from the input: the keyword itself, and three parameter lists. The IFDEF test obviously yields `true', and therefore yodl `pushes back' the true-list

Symbol sym is defined!
DEFINEMACRO(testmac)(0)(Testmac now stands for one thing.)

Next, yodl sees normal text (Symbol sym is etc.). This is sent to the output file.

After this, the DEFINEMACRO statement is seen, which defines testmac() as a placeholder for one thing.

Finally, testmac() is seen. The parser recognizes a user-defined macro having no arguments, which conforms to the definition. The text

Testmac now stands for one thing.

is pushed back.

Finally, yodl reads this last string from its input and sends it to the output.

The most complex part of the program is therefore the tokenizer (lexical analyzer): that part is responsible for matching tokens such as identifiers in the input, and for pushing back text. The pushing back is implemented as a relocatable buffer, which is expanded as text is pushed into it and which shrinks as the lexical analyzer processes input. I already achieved about 100% speed gain by optimizing the tokenizer, but this implementation can still be somewhat of a CPU hog. (But hey, premature optimzation is the root of all evil, says D.E. Knuth. So why optimize, as long as you don't know whether you're being premature about it?)

The lexical analyzer is furthermore responsible for the line continuation feature (see section ??) and for the SUBST mechanism (see section ??). These mechanisms are also implemented with the push-back method.

Go back to index of Yodl.

Please send Yodl questions and comments to yodl@icce.rug.nl.

Please send comments on these web pages to (address unknown)

Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.

This page was built from Yodl-1.31.18 by

<(address unknown)>, Wed May 2 05:35:48 2007 EDT.