% This is a file written using noweb
%
% $Id: fgparse.nw,v 1.8 1998/06/11 10:59:14 felix Exp $
%

\documentclass{article}
\usepackage{a4,version,pretzeldoc,noweb}
\begin{document}

\includeversion{bigspecs}
%\includeversion{impls}
%\excludeversion{bigspecs}% without elaborate specification
\excludeversion{impls}% without implementation



\title{A Generator for the Prettyprinting Parser}
\author{Felix G\"artner}
\maketitle


\begin{abstract}
    This file describes the interface \begin{impls}and
    implementation\end{impls} of a new \CPP\ class that can be used to
    build a prettyprinting parser from a formatted grammar file.
    A formatted grammar file is a file containing a context free
    grammar which is enhanced by additional formatting information.
    The prettyprinting parser constitutes the main part of a
    prettyprinter for the language specified by the formatted grammar.
    This file is used within the framework of the ``pretzel''
    project. Pretzel is a universal prettyprinter generator.
\end{abstract}


\tableofcontents


\section{Introduction}

A {\em prettyprinting parser} is a parser that does prettyprinting
as a result of parsing. Such a parser normally uses an attribute
grammar, i.e.~every token has an associated attribute. These attributes
specify how the token looks. An attribute grammar that contains such
prettyprinting information will be called a {\em formatted grammar}.
Together with information about the formatting of terminal tokens
(so called {\em formatted tokens}) a formatted grammar defines a
prettyprinter.

The ``pretzel'' project aims to generate such a prettyprinter
automatically from a formatted token and formatted grammar description
laid down in files of a special format. These files are called the {\em
formatted token file} and the {\em formatted grammar file}. The
precise format of these files can be obtained by reading the
main documentation of the pretzel project. I will assume that you
know about this and also about the prettyprinting method that the
generated prettyprinter uses.

@ This document describes the interface of a new \CPP\ class that generates
a prettyprinting parser from a formatted grammar file.

This is an overview over the frame the header file:

<<header>>=
#ifndef FGPARSE_H
#define FGPARSE_H

<<Header files to include in the specification>>
<<Declaration of the parser generator class>>

#endif
@

\begin{impls}
  The implementation of this class will be done using the standard
  \UNIX\ tool bison, which is a parser generator. The idea is to parse
  the formatted token file and convert it into a valid bison file, from
  which a parser then can be build (using bison of course).

  So here's the frame of the bison file.

<<bison-implementation>>=
%{
#include <stdlib.h>
#include "fgparse.h"    /* include own header */
#include <assert.h>
#include <sstream>
%}

<<bison definitions>>
%%
<<bison rules>>
%%
<<bison user code>>

@

\end{impls}



\section{Interface (Overview)}

The thing of interest here is the generator for the prettyprinting
parser. The generator is a new \CPP\ class called [[Pparse_builder]].
It has a single member function called [[build]] which does all the
work of building. It takes two arguments:
 %
\begin{enumerate}

\item The first one is a pointer to an input stream where it expects to
read the formatted token file from.

\item The second one is a pointer to an output stream, where it writes
the generated code to.

\end{enumerate}
 %
The function returns an integer value to indicate if everything
worked well (0) or if an error occured (1). For a description on how
the generated prettyprinting scanner looks, see section
\ref{sec:parserinterface} on page \pageref{sec:parserinterface} below.

There are two more functions for error diagnosis. The first is called
[[error_text]] which returns a more elaborate description of why the
[[build]] function might have returned a 1 as a string. The second is
called [[error_line]] which returns the line of the input file
where this error has occured. The [[debug_on]] and [[debug_off]]
functions allow to watch the generation process in details.

<<Declaration of the parser generator class>>=
class Pparse_builder {
public:
  Pparse_builder();
  ~Pparse_builder();
  int build(istream*, ostream*);

  // for error diagnosis and debugging:
  char* error_text();
  int error_line();
  void debug_on();      // write what the generator is doing to cerr
  void debug_off();

private:
  <<Private members of the generator class>>
};

@

These are the header files needed.

<<Header files to include in the specification>>=
#include <iostream>
@


\section{Interface (Details)}

In this section I will present a complete reference of the new
[[Pparse_builder]] class\begin{impls} together with it's
  implementation\end{impls}.

\begin{impls}

  \subsection{Ideas behind the Implementation}

  The idea behind the implementation is to use bison to generate a
  parser and do the translation from formatted grammar file to bison file
  on the fly whilst parsing. The format of the formatted grammar file is
  so similar to a bison file, that this is possible. The parser will
  use the new scanner class [[Fgscanner]] described in the document
  {\tt fgscan.nw}.

  By default bison generates a global parser function [[yyparse]]. We
  want to shield this and all the other globals within the new class.
  So we declare the parser to be pure and redefine the prefixes of the
  globals using the {\tt -p} command line option of bison. We'll use
  the same prefix as used in generating the scanner
  (``[[Pretzel_fg]]'').

<<bison definitions>>=
%pure_parser

%{
#include "fgscan.h"       /* include scanner */
%}
@

  The scanner function expected by [[yyparse]] is called
  [[Pretzel_ftscan]] if we use the {\tt -p} option. But actually it's
  a member function of the scanner class [[Fgscanner]]. How deal with
  this problem?

  We'll define a private member of the [[Pparse_builder]] class to hold
  an instance of the scanner class and use this to scan the input
  file.

<<Private members of the generator class>>=
  Fgscanner* scanner;
@

  We'll also need to include the specification for this class then.

<<Header files to include in the specification>>=
#include "fgscan.h"
@

  So a call to the scanner is like calling the [[scan]] member
  function of the [[scanner]] itself. We need a way to simply trick
  bison into using [[scanner->scan]] everytime it wants to
  call [[Pretzel_ftlex]]. This is more difficult than it seems,
  because [[yyparse]] will be called from inside a member function
  with no idea what scanner object to use.

  We'll get around this problem by using a static variable again for
  communication. It will always hold a pointer to the current scanner
  to be used by [[yyparse]]. This is not very nice but necessary,
  since bison doesn't allow us to redefine the interface to
  [[yyparse]] at all.  This means that the [[build]] member function
  will work well with multiple instances of the [[Pparse_builder]]
  class, but {\em will not be reentrant}. (See also page
  \pageref{yyparseinterface2} for another problem concerning the
  interface of [[yyparse]].)

  \label{yyparseinterface1}

<<bison definitions>>=
%{
static Fgscanner* static_scanner = NULL;
%}
@

  a

  So when Bison calls [[yylex]] it should actually call the following
  small function. It's static, so that it won't upset other Bison
  parsers around, but we have to undefine any macro definition of
  [[yylex]] (such as those introduced when using the [[-P]] option).
  Also, we define the [[YYLEX_DECL]] macro so that it is declared
  okay.

<<bison user code>>=
static int yylex(Attribute* *rval)
{
    return(static_scanner->scan(rval));
}

@
  Some versions of Bison (the newer ones) have a [[YYLEX_DECL]] macro
  that can be used to declare [[yylex]]. We'll declare it ourself and
  make sure it isn't declared again.

<<bison definitions>>=
%{
    /* make sure yylex is declared with name ``yylex'' */
#undef yylex
static int yylex(Attribute* *rvalue);
    /* make sure that it's only declared once */
#define YYLEX_DECL
%}
@


  The semantic value of the tokens manipulated within bison are simple
  strings that are joint together while parsing. Joining together
  strings in the right way will result in a giant string, which is the
  valid bison file, when the parsing has finished.

  As we don't want to get into any hassle copying strings from one
  place to another, we'll use the [[Attribute]] class to do this. We
  won't use the [[Format_attributes]] though, but only the
  [[Text_attributes]]. They weren't written for this task, but they
  are certainly good candidates to be used.

<<bison definitions>>=
%{
#include "attr.h"

#define YYSTYPE Attribute*
%}
@

  So now, when parsing the formatted token file and generating the
  flex file, the parser will finally arrive at the end of its work and
  have a huge attribute that contains the contents of the flex file.
  Now what?  Well, we can't output it to the output stream specified
  in the [[build]] function, since there is no way to tell [[yyparse]]
  what output stream to take. Also, we can't use a private member of
  the builder to hold this attribute temporaily, because the
  [[yyparse]] function doesn't have access to it, even if we declare
  it friend to the builder class. (This is because the [[yyparse]]
  function has no means of knowing, which object it was invoked by.)
  So the best thing to do is to store this huge
  attribute in a static variable called [[rest]] and output it later,
  when [[yyparse]] has retuned. (See also page
  \pageref{yyparseinterface} for another problem concerning the
  interface of [[yyparse]].)

  \label{yyparseinterface2}

<<bison definitions>>=
%{
static Attribute* rest = NULL;
%}
@

  A static variable within this module will hold the current text
  describing the error message encountered. Normally this will be a
  syntax error and this is the default entry. But while parsing, we
  might be able to specify more precisely what's gone wrong. In such
  cases we simply assign a new text to this variable.

<<bison definitions>>=
%{
static char* error_message = "syntax error";
%}
@

  To be able to debug the grammar, I've put in the [[debug_on]] and
  [[debug_off]] member functions into the class. They'll simply switch
  on (and off) the bison debugging mechanism. The semantic values of
  the tokens will be output to [[cerr]] too, and so we need to include
  the header of the [[Output]] class too.

<<bison definitions>>=
%{
#define YYDEBUG 1
#include "output.h"
%}
@

\end{impls}




\subsection{Creating and Destroying the Generator}

The constructor and destructor of the [[Pparse_builder]] class do
nothing much. They simply have to be there before you call [[build]].

\begin{impls}

  The implementation is straightforward too.

<<bison user code>>=
Pparse_builder::Pparse_builder()
{
    scanner = NULL;
};
Pparse_builder::~Pparse_builder()
{
    if (scanner != NULL) delete scanner;
};
@

\end{impls}



\subsection{Activation and Results of the Generator}
\label{sec:parserinterface}

The actual work is performed when you call [[build]]. It takes a
pointer to an input stream [[is]] and a pointer to an output stream
[[os]]. It then starts to read a formatted grammar file from [[is]] and
outputs code for the prettyprinting parser to [[os]].

The resulting code is bison code that contains the complete definition
of a new \CPP\ prettyprinting parser class. This new class is a
subclass of an abstract base class [[Pparse]], that offers one virtual
function [[prettyprint]] that is redefined within the newly generated
subclass.

The exact interface of the generated prettyprinter class can be found
in the file {\tt Pparse.h}. It is:

\label{pretparsespecs}

<<abstract base class for [[Pparse]]>>=
#include<iostream>
#include"attr.h"
#include"output.h"

class Pparse {
public:
  Pparse() {};
  virtual ~Pparse() {};
  virtual int prettyprint(istream*, ostream*) = 0;
  virtual int prettyprint(istream*, Output*) = 0;
};
@

The interface of the generated prettyprinter is:

<<Interface of the generated parser>>=
class PPARSE_NAME : public Pparse {
public:
  PPARSE_NAME();
  ~PPARSE_NAME();
  int prettyprint(istream*, ostream*);
  int prettyprint(istream*, Output*);

  void debug_on();
  void debug_off();
};
@

The name of the generated class is [[PPARSE_NAME]] by default and has
to be redefined using preprocessor macros, such as for example
 %
\begin{quote}
  [[#define PPARSE_NAME Prettyprinter_for_pascal]]
\end{quote}
 %
By default the name of the derived subclass will be [[Ppparse]] (note
the three `p's).

A header file containing the abstract base class would look like this:

<<header file for abstract base class of [[Pparse]]>>=
#ifndef PPARSE_H
#define PPARSE_H

<<abstract base class for [[Pparse]]>>

#endif
@

A header file containing the derived parser class could look like
this (by default the header file is called {\tt Ppparse.h}). Note that
there is no precaution against double inclusion.

<<header file for prettyprinting parser>>=
/* header file for a prettyprinting parser */

/* NB: This file is NOT protected against double inclusion! */

#include "Pparse.h" // include abstract base class

#ifndef PPARSE_NAME
#define PPARSE_NAME Ppparse
#endif

<<Interface of the generated parser>>
@

\begin{impls}

  The implementation of the [[build]] function is possibly the
  trickiest thing in this entire class. It uses the internal
  [[Fgscanner]] class object and then calls [[yyparse]].

  The assignment [[static_scanner = scanner]] before calling
  [[yyparse]] ensures that the right scanner is used. (See also page
  \pageref{yyparseinterface1} for a description of this problem.)

  We'll have to use a nonstandard [[Output]] for outputting the file
  contents, as the abstract base class can't be used. You can use any
  output subclass that doesn't upset the strings (and
  [[Latex_cweb_output]] is a good candidate for this, especially
  because it comes free with the abstract base class).

<<bison user code>>=
int Pparse_builder::build(istream* is, ostream* os)
{
  scanner = new Fgscanner(is);
  assert(scanner != NULL);
  <<prepare additional text for the generated file>>
  static_scanner = scanner;
  int ret_val = yyparse();
  if (ret_val != 0) {
    return(1);
  } else {
    Latex_cweb_output tmp_output(*os);   // create an output for [[os]]
    rest->print(tmp_output);  // and print the contents there.
    return(0);
  }
}

@

To ensure that the resulting bison code gets right, a lot of
declarations and additional \CPP\ code have to be added to the file.
This is done by declaring some additional strings that get appended to
the front, middle and back of the bison file when it is produced.
These additional lines essentially contain the rest of the \CPP\
implementation of the interface specified above.

  There are three static variables holding text:

  \begin{itemize}
  \item [[initial_lines]] will hold text that is put to the very
    beginning of the resulting bison file.
  \item [[before_lines]] will hold text that is put just before the
    \verb+%%+ line that starts the rules part of the bison file.
  \item [[after_lines]] will be put behind a second \verb+%%+ line and
    will contain additional \CPP\ code.
  \end{itemize}

  The corresponding integer variables will be used to turn these
  string arrays into [[Attribute]]s, which in turn are held in the
  corresponding attribute variables.

<<bison definitions>>=
%{
static
char *initial_lines[] = {
        "/**************************************************    \n",
        " * This is a prettyprinting parser implementation *    \n",
        " * file generated by pretzel. (It's Bison code!)  *    \n",
        " *                                                *    \n",
        " *************************************************/    \n\n",
  <<contents of [[initial_lines]]>>
        "/*************************************************     \n",
        " * various user definitions start here:          *     \n",
        " *************************************************/    \n\n"
};
static
char *before_lines[] = {
        "/*************************************************     \n",
        " * some pretzel definitions start here:          *     \n",
        " ************************************************/     \n\n",
  <<contents of [[before_lines]]>>
  "\n"
  };
static
char *after_lines[] = {
        "/*************************************************     \n",
        " * additional pretzel code starts here:          *     \n",
        " *************************************************/    \n\n",
        <<contents of [[after_lines]]>>
        "/*************************************************     \n",
        " * optional user defined C code follows here:    *     \n",
        " *************************************************/    \n\n"
  };

static
const int initial = sizeof(initial_lines)/sizeof(initial_lines[0]);
static
const int before = sizeof(before_lines)/sizeof(before_lines[0]);
static
const int after = sizeof(after_lines)/sizeof(after_lines[0]);

static Attribute *initial_text = create(null);
static Attribute *before_text = create(null);
static Attribute *after_text = create(null);
%}
@

All these lines are accumulated into the three static variables and
are inserted into the file within the grammar specified in section
\ref{sec:grammarrules} on page \pageref{sec:grammarrules}. Using the
constants like [[after]] spares us from counting the line numbers
every time we change the text. Thanks to Andreas Kirschbaum for
suggesting this hack.

<<prepare additional text for the generated file>>=
        int i=0;
        for (i=0; i<initial; i++) {
                initial_text =
                        join(initial_text,
                                create(initial_lines[i])
                        );
        }
        for (i=0; i<before; i++) {
                before_text =
                        join(before_text,
                                create(before_lines[i])
                        );
        }
        for (i=0; i<after; i++) {
                after_text =
                        join(after_text,
                                create(after_lines[i])
                        );
        }
@

The contents of the line arrays contain the actual definitions
(i.e.~the code) of the prettyprinting parser. This code has to obey
the specifications from above (see page \pageref{pretparsespecs}).

The [[initial_lines]] contain the overall includes of headers,
redefinitions of [[YY_INPUT]] and static variables.

<<contents of [[initial_lines]]>>=
        "%{							\n\n",
        "  /* this is needed for the yyprint function below */	\n",
        "#include<stdio.h>					\n\n",
        "  /* include the attribute definitions */		\n",
        "#include\"attr.h\"					\n\n",
        "  /* include the output definitions */			\n",
        "#include\"output.h\"					\n\n",
	"  /* enable parser debugging */			\n",
	"#define YYDEBUG 1					\n\n",
        "  /* redefine the semantic value */			\n",
        "#define YYSTYPE Attribute*				\n\n",
        "  /* enable calls of YYPRINT */			\n",
        "#define YYPRINT(file, type, value) yyprint (file, type, value)\n\n",
        "  /* this is the prototype for yyprint */		\n",
        "static void						\n",
        "yyprint(FILE *file, int type, YYSTYPE value);		\n\n",
	"  /* make sure that our local copy of yylex is		\n",
	"   * declared with name `yylex' and as a static	\n",
	"   * function. This should trick the parser into	\n",
	"   * calling the scanner object stored in		\n",
	"   * `static_scanner' (see below). */			\n\n",
	"  /* undefine yylex to make sure it isn't renamed	\n",
	"   * by any command line option (e.g. -P). */		\n",
	"#undef yylex						\n",
	"  /* declare our local `dummy' yylex */		\n",
	"static int yylex(YYSTYPE *rvalue);			\n",
	"  /* make sure any newer versions of Bison don't	\n",
	"   * redeclare it differently */			\n",
	"#define YYLEX_DECL					\n",
        "%}							\n\n",
@

Within the [[before_lines]] we have to put everything that can be
redefined by the user and has to be mentioned before the body of
[[yyparse]] comes along.

<<contents of [[before_lines]]>>=
        "%{							\n\n",
	"#include <stdlib.h>					\n",
	"  /* include own header only in default case: */	\n",
	"#ifndef PPARSE_NAME					\n",
	"#include \"Ppparse.h\"					\n",
	"#endif							\n\n",
        "  /* this is a definition of the yyerror routine */	\n",
        "int yyerror (const char *s)				\n",
        "{  /* ignore error messages by BISON */		\n",
        "   /* trick! use s do get around warning `s not used'. */\n",
        "   if (1==0) fprintf(stderr, \"%s\", s);		\n",
        "   return 0;						\n",
        "}							\n\n",
        "  /* enable parser debugging */			\n",
        "#define YYDEBUG 1					\n\n",
        "  /* variable to hold the parsed attribute */		\n",
        "static Attribute *rest=create(null);			\n\n",
        "  /* include the scanner only in the default case: */	\n",
        "#ifndef PSCAN_NAME					\n",
	"#define PSCAN_NAME Ppscan				\n",
        "#include \"Ppscan.h\"					\n",
        "#endif							\n\n",
	"  /* trick the yyparse function to use the scan member	\n",
	"   * of the scanner class: */				\n",
	"static Pscan* static_scanner = NULL;			\n",
	"static int yylex( YYSTYPE *rvalue )			\n",
	"{							\n",
	"    return(static_scanner->scan(rvalue));		\n",
	"}							\n\n",
        "%}							\n\n",
        "%pure_parser						\n\n",
@

The [[after_lines]] contain the rest of the definition of the
implemented class.

The actual prettyprinter previously did as many calls to [[yyparse]]
as were necessary to reach [[eof]]. Now, [[yyparse]] is called
exactly once. This makes a description of the prettyprinter semantics
to the outside world easier (``input must be fully parseable'').
However, we still empty the input stream up to [[eof]] to keep
the semantics of the old prettyprinter d'accord with the new one.

<<contents of [[after_lines]]>>=
        "#ifndef PPARSE_NAME					\n",
        "#define PPARSE_NAME Ppparse				\n",
        "#endif							\n\n",
	"PPARSE_NAME::PPARSE_NAME()				\n",
	"{							\n",
	"}							\n",
	"PPARSE_NAME::~PPARSE_NAME()				\n",
	"{							\n",
	"    static_scanner = NULL;				\n",
	"}							\n",
	"int PPARSE_NAME::prettyprint(istream* is, ostream* os)	\n",
	"{							\n",
	"    Latex_cweb_output ppoutput(*os);			\n",
	"    return(prettyprint(is, &ppoutput));		\n",
	"}							\n",
	"int PPARSE_NAME::prettyprint(istream* is, Output* outp)\n",
	"{							\n",
	"    int ret_val = 0;					\n",
	"    PSCAN_NAME ppscanner(is);				\n",
	"    static_scanner = &ppscanner;			\n",
	"    ret_val=yyparse();					\n",
        "    while (!(is->eof())) {				\n",
        "        char c;					\n",
	"        is->get(c);					\n",
        "    }							\n",
        "    rest->print(*outp);				\n",
        "    return(ret_val);					\n",
        "}							\n\n",
	"void PPARSE_NAME::debug_on() { yydebug = 1; }		\n",
	"void PPARSE_NAME::debug_off() { yydebug = 0; }		\n\n",
        "  /* This is a routine to print the semantic value */\n",
        "static void \n",
        "yyprint(FILE *file, int type, YYSTYPE value) {\n",
        "  Latex_cweb_output os(cerr);\n",
        "  value->print(os);\n",
        "}\n",
@

\end{impls}


\subsection{Error Diagnosis and Debugging}

The error diagnosis functions can be used, if the [[build]] function
has returned an error code (1). Calling [[error_text]] will result in
a more elaborate explanation of the error cause (mostly ``syntax
error'') and [[error_line]] will reveal the line number of the
formatted token file, where the generator choked.

\begin{impls}

  The implementation is straighforward again. The [[error_text]]
  function simply returns the internal error message and the
  [[error_line]] function askes the scanner and returns its line
  value.

<<bison user code>>=
char *Pparse_builder::error_text()
{
  return(error_message);
}
int Pparse_builder::error_line()
{
  return(scanner->line());
}
@

The debugging member functions \ldots % XXX

\end{impls}





\begin{impls}

\section{Bison Implementation of the Generator}
\label{sec:grammarrules}

This section contains the complete bison rules for the generator
function [[yyparse]]. It is the main part of the implementation. All
this was written for GNU bison version 1.19.

\subsection{Token Declarations}

Bison will expect the scanner to return the following tokens:

<<bison definitions>>=
%token ID
%token DOLLARS
%token STRING
%token FMTCMD
%token A_LINE
%token PERCENT_PERCENT
%token UNKNOWN
%token BEG_CODE
%token CODE
%token END_CODE
@

\subsection{The Grammar Rules}

Now we'll go through the grammar rules one by one in a top down
fashion.

A formatted grammar file consists of lots of top lines, a final top
line (containing `\verb+%%+') and a list of productions that may be
followed by again a `\verb+%%+' and another block of lines. If we want
to insert stuff into the definitions section of the output, we have to
do this between [[$1]] and [[$2]] of the join command below.

<<bison rules>>=
prebison_file   : lines PERCENT_PERCENT prod_list       {
                        rest=$$=join(initial_text,$1,
                                before_text,$2,$3,
                                create("\n%%\n"),after_text);}
                | lines PERCENT_PERCENT prod_list PERCENT_PERCENT lines {
                        rest=$$=join(initial_text,$1,
                                before_text,$2,$3,$4,$5,after_text);}
;

lines           : /* empty */           {rest=$$=create(null);}
                | lines A_LINE          {rest=$$=join($1,$2);}
;

@

Here is the definition of the production list. It may be empty!

<<bison rules>>=
prod_list       : /* empty */           {rest=$$=create(null);}
                | prod_list prod        {rest=$$=join($1,$2,create("\n"));}
;
@

Now we will see, what a production can be: A production starts with an
[[ID]] followed by an `[[:]]'. The tail of a production can vary, but
it has to be ended with a `[[;]]'.

<<bison rules>>=
prod            : ID ':' prod_tail ';'  {rest=$$=join(
                                                create("\n"),
                                                $1,
                                                create("\t\t: "),
                                                $3,
                                                create("\n;")
                                                );}
;
@

The tail of a production can contain a long (or possibly empty)
list of right sides.

<<bison rules>>=
prod_tail       : right_side_list
;
@

The definition of the [[right_side_list]] ensures, that several right
sides of rules may be concatenated with `\verb+|+'.

<<bison rules>>=
right_side_list : right_side    {rest=$$=$1;}

                | right_side_list '|' right_side        {rest=$$=
                                join(
                                        $1,
                                        create("\n\t\t| "),
                                        $3
                                );}
;

@

If the right side of the rule is empty, a null-attribute will be the
attribute of the right side. This is sometimes a little misleading,
because this [[null]] command is not visible (neither physically nor
in the documentation). Semantic values should be really empty. But I
know of no easy way to resolve this problem. In fact, I can't figure
out how to do it without returning a special marker and having the
parser check every semantic value before [[join]]ing.

<<bison rules>>=
right_side      : /* empty */   {rest=$$=create(
                        "/* empty */\t\t {rest=$$=create(null);}"
                                        );}
;
@

If the right side contains nothing but an [[id_list]], the default
mechanism darts into action. The corresponding attribute consists
simply of a list of all the attributes of the right side of the rule.
Note that a list is not needed if only one identifier sleeps on the
right side. Then we can say:
 %
\begin{quote}
   [[rest=$$=$1;]]
\end{quote}
 %
But the join function works also with only one parameter
so we can say
 %
\begin{quote}
   [[rest=$$=join($1);]]
\end{quote}
 %
and make the code a little simpler.

Note also that [[list_length]]$\geq 1$ if an [[id_list]] has
been parsed.

I have built in a mechanism, so that [[join]] is only invoked with
a maximum of two arguments. This mechanism works as follows:

\begin{itemize}
\item Every time a new [[$i]] is added, the string
    %
  \begin{quote}
    [["join($i,"]]
  \end{quote}
    %
  is added to the [[ostringstream istr]]. Simultaneously a ``[[)]]'' is
  added to a second [[ostringstream jstr]]. So every ``[[(]]'' has a
  matching ``[[)]]'' in [[jstr]]. This is done as long as there is
  still at least one [[$i]] to add.
\item The last [[$i]] is added simply as
  \begin{quote}
    [["$i"]]
  \end{quote}
\item The resulting attribute is created by simply concatenating
  \begin{quote}
    [["\t\t{rest=$$="]]
  \end{quote}
  with [[istr.str()]] and [[jstr.str()]] and
  \begin{quote}
    [[";}"]]
  \end{quote}
\end{itemize}

<<bison rules>>=
right_side : id_list            {
                                std::ostringstream istr;
                                std::ostringstream jstr;

                                for (int i=1; i<list_length; i++) {
                                        istr << "join($" << i << ",";
                                        jstr << ")";
                                }
                                istr << "$" << list_length;

                                istr << ends; /* end the two stringstream */
                                jstr << ends;

                                rest=$$=join(
                                        $1,
                                        create("\t\t{rest=$$="),
                                        create(istr.str().c_str()),
                                        create(jstr.str().c_str()),
                                        create(";}")
                                        );
                                /* create allocates storage for the string */
                                /* so the destructor for istr can do his */
                                /* work without harm */
                                /* after creating the list, we set */
                                /* list_length back to zero, because the */
                                /* next list is empty again. */
                                list_length=0;
                                }
;
@

The [[list_length]] is a static variable. Also, we have to include the
{\tt sstream} header, because we've used an [[ostringstream]].

<<bison definitions>>=
%{
#include<sstream>
static int list_length=0;
%}

@

Of course you are also allowed to have an empty [[id_list]].

<<bison rules>>=
right_side : attr_def           {rest=$$=join(
                                        create("/* empty */"),
                                        create("\t\t\t"),
                                        $1
                                        );}
;

@

The normal case sees an [[id_list]] with an attribute
definition at the end. From version 2.0 of \pretzel{} onwards we'll
also allow code fragments to be inserted at the beginning of, within or
at the end of the attribute definition. Initial and ending code is
optional, but an attribute definition is necessary if you want to
specify either or.

The initial code will be placed in front of the [[$$]] assignment,
ending code behind it. Code fragments in the middle are expected to
return [[Attribute*]] and are simply [[join]]ed together like
everything else.

<<bison rules>>=
right_side : id_list opt_code
	'{' attr_def '}' opt_code	{rest=$$=join(
						$1,
						create("\t\t{ "),
						$2,
						create("\n\t\t\t"),
						create("rest=$$=\n"),
						$4,
						create(";\n\t\t"),
						$6,
						create("\n\t\t}"));}
;
@

Let's say, what [[code]] and [[opt_code]] are first.

<<bison rules>>=
code    : BEG_CODE code_lines END_CODE     { rest=$$=$2; }
;

code_lines : CODE                          { rest=$$=$1; }
        | code_lines CODE                  {
                rest=$$=join($1, create("\t\t\t"), $2);
                }
;

opt_code : code                            { rest=$$=$1; }
         | /* empty */                     { rest=$$=create(null); }
;

@



[[list_length]] counts the number of identifiers on the
right side of the rule to allow a correct number of
[[$i]]'s in the default rule, that is if the attribute
definition is missing.

<<bison rules>>=
id_list : ID                    {rest=$$=$1; list_length=1;}
        | id_list ID            {rest=$$=join($1,create(" "),$2);
                                        list_length++;
                                }
;

attr_def        : attr_def_list         {
                                        rest=$$=join(
                                                create("\t\t\t"),
                                                $1
                                                );}
;
@

Note that the [[attr_def_list]] must contain at least one element
Also, in version 2.0 of pretzel we'll allow to omit the [[+]].

<<bison rules>>=
attr_def_list   : attr          {rest=$$=$1;}

                | attr_def_list '+' attr        {
                                rest=$$=join(
                                        create("join(\n\t\t\t\t"),
                                        $1,
                                        create(",\n\t\t\t\t"),
                                        $3,
                                        create(")")
                                        );}
                | attr_def_list attr        {
                                rest=$$=join(
                                        create("join(\n\t\t\t\t"),
                                        $1,
                                        create(",\n\t\t\t\t"),
                                        $2,
                                        create(")")
                                        );}
;

attr    : DOLLARS               {rest=$$=$1;}
;

@

The scanner returns the name of the format-command as semantic value.
This command shall be created in the output file. Therefore we must
say:
 %
\begin{quote}
  create this:    [["create(" + name of the format command + ")"]]
\end{quote}
 %
This yields:
 %
\begin{quote}
  [[join(create("create("),$1,create(")"))]]
\end{quote}

<<bison rules>>=
attr    : FMTCMD                {rest=$$=join(
                                        create("create("),
                                        $1,
                                        create(")")
                                        );}
;

@

A similar thought goes with strings. The scanner returns
the contents of the string packed up nicely into double
quotes. The only thing the parser has to to is:
 %
\begin{quote}
  create this:   [["create(" + string in double quotes + ")"]]
\end{quote}
 %
This can be said as follows:
 %
\begin{quote}
  [[join(create("create("),$1,create(")"))]]
\end{quote}

<<bison rules>>=
attr    : STRING                {rest=$$=join(
                                        create("create("),
                                        $1,
                                        create(")")
                                        );}
;
@

An attribute can be code too, at least from version 2.0 of \pretzel{}
onwards.

<<bison rules>>=
attr	: code			{ rest=$$=$1; }
;
@



\section{Additional bison Functions}

Here's the definition of the [[yyerror]] routine needed by the
generated parser. It isn't supplied automatically, so we have to cater
for it. We don't want to output any bison error messages so we simply
return 0. Its name has to contain the same prefix as specified on the
bison command line.

<<bison user code>>=
int Pretzel_fgerror (const char *s)
{
        return 0;
}
@

This line is needed so that Bison doesn't declare the supposed
[[yyerror]] routine extern.

<<bison definitions>>=
%{
int Pretzel_fgerror (const char *s);
#define YYERROR_DECL
%}
@

\end{impls}



\begin{bigspecs}

\section{Examples}

This section contains some examples of how to use the [[Pparse]] class
and subclasses.

[\ldots]

\end{bigspecs}

\end{document}


%
% $Log: fgparse.nw,v $
% Revision 1.8  1998/06/11 10:59:14  felix
% changed reference is.get to is->get of course. Typical example of
% bug introduced by changes.
%
% Revision 1.7  1998/04/25 09:03:18  gaertner
% Added a note on empty rules to documentation.
% Changed generated prettyprinter code to do exactly one parse.
% This makes sematics in error cases easier.
%
% Revision 1.6  1996/09/17  13:45:40  gaertner
% Changed the decaration and definition of yylex in the generated
% parser so that it works with g++ and diferent versions of Bison.
%
% Revision 1.5  1996/09/12  18:44:10  gaertner
% Refined declarartion handling of yylex and yyerror.
%
% Revision 1.4  1996/09/12  18:00:55  gaertner
% Changed the yylex function to static. Hope the trick of having
% yyparse call the local yylex works now. I've taken care that
% several macros are defined okay (e.g. yylex and YYLEX_DECL).
%
% Revision 1.3  1996/04/22  09:18:37  gaertner
% Allows user code in attribute definitions now.
%
% Revision 1.2  1996/03/21  11:04:23  gaertner
% Added epty implementations to Pparse constructor and destructor.
% Refined generated output so it will compile.
%
% Revision 1.1  1996/03/21  08:13:57  gaertner
% Initial revision
%
%


