%
% $Id: chap3.tex,v 1.4 1995/04/20 13:13:03 gaertner Exp gaertner $
%
% $Log: chap3.tex,v $
% Revision 1.4  1995/04/20  13:13:03  gaertner
% Corrections proposed by JS
%
% Revision 1.3  1995/02/23  12:20:14  gaertner
% Final revision (including index).
%
% Revision 1.2  1995/02/21  09:24:22  gaertner
% Replaced \mkWEAVE with \pretzel.
%
% Revision 1.1  1995/02/21  09:21:47  gaertner
% Initial revision
%
%



\chapter{Literate Programming and \pretzel}
\label{three}

The \pretzel\ program is the result of a project that the author
worked on in 1994 at the Technical University of Darmstadt. It is a
small contribution to a larger project of research that has to do with
the literate programming paradigm that I have already mentioned in the
introduction to this report.

This chapter now introduces the literate programming paradigm in
slightly more detail and outlines the research project of which
\pretzel\ is only a small part. Taking this larger frame into
account, I will finally describe the role that \pretzel\ could
play in this research.



\section{On Literate Programming}

``When was the last time you spent a pleasant evening in a comfortable
chair, reading a good program?'' asks Jon Bentley in the introduction
to his regular column in the Communications of the ACM
\cite{Bentley:CACM-29-5-364}. As might be suspected, Jon Bentley is
far from being sarcastical; his question is meant seriously and our
not taking him seriously reveals only the poor attitude we have
towards the readability of a program's source code.

\index{Bentley, Jon}\index{ACM}\index{Sarcasm}

But how can it be pleasant and amusing to read a program, if this
program is simply another sequence of programming language
commands destined for a compiler, that---if we are lucky---may be
decorated with a few lines of commentary? This surely isn't very
pleasant; but the reading and understanding of a program is far too
important to be unpleasant. The only way to get out of this
dilemma is to write programs with a different attitude: Programs
are not merely lines of code that primarily instruct a computer to
act in a special way; programs must be destined to meet the eye of
a {\em human being} and explain him or her what the author wants a
computer to do \cite[p.~97]{Knuth:CJ-27-2-97}.

Though this necessity has long been recognized, it is due to
Donald Knuth and his idea of ``literate programming'' that today
programs can be written in a style that makes comprehension of
source code indeed a pleasant and rewarding job. The name of his
new concept is derived from the idea, that programs should be
considered {\em works of literature\/}; in the course of this idea,
the programmer's main concerns switch from efficiency and brevity
to exposition and excellence of style (without forgetting the
first two goals of course). Considering this turning of approach,
Knuth's idea can legitimately be called a new {\em programming
paradigm} \cite{Cordes:COMPUTER-24-6-52}.

\index{Knuth, Donald E.}\index{works of literature@``works of literature''}
\index{Programming paradigm}\index{Exposition and excellence of style}
\index{Literate programming}

There has been (and still is) a lot of discussion concerning the
question of what makes a program literate or what languages really
support a literate programming style. However, there seems to be a
consensus that literate programming languages must enable the
programmer to be absolutely free in choosing the order and way of
presenting his ideas. It must be possible to use a top-down approach
of presentation as well as a bottom-up approach or even a mixture of
both. The key notion here is that it must support arbitrary {\em
refinements} together with an easy way to insert comments into the
program (imagine the notion of delimiting parts of the source code
instead of bracketing comments, as you would normally do in
traditional languages).  The only restriction the programmer should
have to worry about is the limitation of his own programming skills.

\index{Refinements}

In 1984 Knuth presented a first literate programming language that was
based on \PASCAL, which he called \WEB. In fact, he didn't introduce a
whole new language: \WEB\ programs and \PASCAL\ programs are
essentialy the same thing, only that the \PASCAL\ source code is split
into small pieces and shuffeled around in a \WEB\ program.  Knuth
proved that his \WEB\ system was practicable by writing (among other
smaller examples) a full scale software system in it, namely
\TeX~\cite{Knuth:ct-b,TeXbook}. These two things, together with his
naming of the area, are Knuth's main contributions to the idea of
literate programming.

\index{web@\WEB}\index{pascal@\PASCAL}\index{tex@\TeX}


\subsection{The \WEB\ system}

A conventional \WEB\ system---like that first presented by Knuth
\cite{Knuth:CJ-27-2-97}---is chiefly a combination of two other languages:
 %
\begin{itemize}

\item a document formatting language and

\item a programming language.

\end{itemize}
 %
Knuth's initial \WEB\ used \TeX\ as document formatting language
and \PASCAL\ as programming language, but the idea is chiefly
independent of the underlying languages.

A literate program is a combination of these two languages:
commentary is written in the document formatting language, source
code is written in the programming language. The \WEB\ system
itself only provides a small set of commands that allow the user
to switch between these two languages and that identify the actual
order of the source code making it possible for a normal compiler
to handle it.

As implied by the last sentence, one main part of the \WEB\ system is
a routine that converts a literate program into a form, that a
conventional compiler may understand; the converted code then may be
compiled into a machine-executable program. This way of processing is
called {\em tangling the web} and the corresponding system routine is
called \TANGLE. The second main line of processing converts the \WEB\
into a high quality document that serves as a source code
documentation. This line is called {\em weaving the web}, the system
routine perfoming this task consequently \WEAVE. These two lines of
processing are visualized in figure \ref{web-system}.

\begin{figure}
% put picture here
\begin{center}
  \unitlength1cm
\begin{picture}(12,7)
\put(4.5,5.5){\framebox(2,1){{\tt .WEB}}}
\put(2,3.5){\framebox(1.5,1){{\tt .PAS}}}
\put(2,1){\framebox(1.5,1){{\tt .EXE}}}
\put(7.5,3.5){\framebox(1.5,1){{\tt .TEX}}}
\put(7.5,1){\framebox(1.5,1){{\tt .DVI}}}
\put(4.5,5.5){\vector(-3,-2){1.5}}
\put(2.5,5.2){\TANGLE}
\put(3,3.5){\vector(0,-1){1.5}}
\put(1,2.7){\parbox{19mm}{PASCAL-Compiler}}
\put(6.5,5.5){\vector(3,-2){1.5}}
\put(7.5,5.2){\WEAVE}
\put(8,3.5){\vector(0,-1){1.5}}
\put(8.5,2.7){\parbox{3cm}{\TeX}}
\put(1,0.4){\large Program}
\put(8,0.4){\large Document}
\end{picture}
\end{center}
\caption{\label{web-system}The two lines of processing in Knuth's initial
\WEB\ system.}
\end{figure}


As stated above, the idea of \WEB\ is basically language independent.
The past few years have seen many new \WEB\ systems for a lot of
different combinations of languages. Many of them have been inspired
copies of Knuth's initial \PASCAL-\WEB, like for example
\CWEB\ \cite{Knuth:CSS93} for \C, \CPP\ and \TeX. As mentioned earlier there
has also been a lot of disagreement on what exactly a \WEB\ system
must be able to do. This is mostly due to the language dependency of
the weaving process.

Today one can see three main categories of \WEB\ systems:

\begin{enumerate}

\item In this category you would find Knuth's initial \WEB\
system. The structure of the underlying languages are hard-wired
into the implementation and the output (mostly in \TeX) is of high
quality and sees the program code beautifully prettyprinted.
Moreover an index of identifiers is generated as well as a complex
cross-reference of sections.

\index{Prettyprinting}

\item Here, the language dependencies are excluded from the actual
\TANGLE\ and \WEAVE\ processors and put into modules. When writing
a literate program the user can switch between programming
languages and formatting languages by choosing the appropriate
modules. An example for such a \WEB\ system is \t{FWEB} by John
Krommes \cite{Krommes:fortran-web}.

\index{Krommes, John}\index{fweb@\t{FWEB}}

\item This category collects all the \WEB\ systems that are more
or less programming language independent (\LaTeX\ is mostly used
as document formatting language). Due to this independency, these
\WEB\ systems have no prettyprinting feature and are not able to
generate indexes or cross references. Examples for these kinds of
\WEB{}s are \t{funnelweb} \cite{Williams:FUM92}, \t{noweb}
\cite{Ramsey:LPT92} and \t{nuweb} \cite{Briggs:NUWEB-93}.

\index{latex@\LaTeX}\index{Funnelweb}\index{Noweb}\index{Nuweb}

\end{enumerate}

There is also a generator for \WEB\ systems called \t{SpiderWEB}
written by Norman Ramsey \cite{Ramsey:CACM-32-9-1051} that produces a
category~one-\WEB. The \pretzel\ program was inspired very much by
this approach (and this report also owes its title to Ramsey's paper).
However, \pretzel\ does not produce a whole functional \WEAVE\
program, but generates only a prettyprinter. But because the
prettyprinting is the most intricate part of the systems of categories
one and two, it would not be a major problem to construct a \WEB\
system around it.  But as \pretzel\ produces a ``prettyprinting
block'' for a specific programming language, the resulting \WEB\
system would rather belong to category two.

\index{Spiderweb}\index{Ramsey, Norman}

The name ``\WEB'' has different origins.  Knuth writes: ``I chose the
name \WEB\ partly because it was one of the few three-letter words of
English that hadn't already been applied to computers.''
\cite{Knuth:CJ-27-2-97} Another source Knuth refers to is the name of the
mother of Knuth's wife, Wilda Ernestine Bates \cite[p.~2]{Knuth:web83}.
Seriously, the name of the system and its subcomponents \TANGLE\ and
\WEAVE\ is a reminiscence to the popular english proverb:
 %
\index{Bates, Wilda Ernestine}\index{Knuth, Donald E.}\index{web@\WEB!name of}
 %
\begin{quote}
Oh, what tangled web we weave

When first we practice to decieve.
\end{quote}




\subsection{The Literate Programming Paradigm and the Software Life Cycle}

The \WEB\ system was originally designed ``for system programmers,
not for high school students or for
hobbyists''~\cite[p.~111]{Knuth:CJ-27-2-97}. This implies that
the literate programming paradigm concentrates on the development and
the documentation of actual system
code~\cite[p.~55]{Cordes:COMPUTER-24-6-52}. So when reviewing the
software life cycle is becomes apparent, that literate programming
covers only parts of it, namely the implementation and additionaly
the final stage of maintenance (see figure \ref{life-cycle}).

\index{Life cycle}\index{Software life cycle}

\begin{figure}
% put picture here
\begin{center}
  \setlength{\unitlength}{0.012500in}%
%
\ifx \SetFigFont\undefined
  \begingroup\makeatletter
  % extract first six characters in \fmtname
  \def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
  \expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
  \ifx\x\y   % LaTeX or SliTeX?
  \gdef\SetFigFont#1#2#3{%
    \ifnum #1<17\tiny\else \ifnum #1<20\small\else
    \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
    \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
       \huge\fi\fi\fi\fi\fi\fi
    \csname #3\endcsname}%
  \else
  \gdef\SetFigFont#1#2#3{\begingroup
    \count@#1\relax \ifnum 25<\count@\count@25\fi
    \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
    \expandafter\x
      \csname \romannumeral\the\count@ pt\expandafter\endcsname
      \csname @\romannumeral\the\count@ pt\endcsname
    \csname #3\endcsname}%
  \fi
  \endgroup
\fi
\begin{picture}(330,75)(30,690)
\thinlines
\put( 99,765){\line( 0,-1){ 45}}
\put(141,765){\line( 0,-1){ 45}}
\put(225,765){\line( 0,-1){ 45}}
\put(273,765){\line( 0,-1){ 45}}
\put(141,690){\line( 0, 1){  6}}
\put(141,696){\line( 1, 0){ 81}}
\put(222,696){\line( 0,-1){  6}}
\put( 30,720){\framebox(330,45){}}
\put(276,690){\line( 0, 1){  6}}
\put(276,696){\line( 1, 0){ 84}}
\put(360,696){\line( 0,-1){  6}}
\put(297,702){\makebox(0,0)[lb]{\smash{\SetFigFont{10}{12.0}{rm}? ? ?}}}
\put( 37,735){\makebox(0,0)[lb]{\smash{\SetFigFont{10}{12.0}{rm}Specification}}}
\put(280,735){\makebox(0,0)[lb]{\smash{\SetFigFont{10}{12.0}{rm}Maintenance}}}
\put(233,735){\makebox(0,0)[lb]{\smash{\SetFigFont{10}{12.0}{rm}Testing}}}
\put(148,735){\makebox(0,0)[lb]{\smash{\SetFigFont{10}{12.0}{rm}Implementation}}}
\put(104,735){\makebox(0,0)[lb]{\smash{\SetFigFont{10}{12.0}{rm}Design}}}
\end{picture}
\end{center}
\caption{
\label{life-cycle}
Life-cycle coverage offered by the
literate paradigm%
%\cite[p.~57]{Cordes91}
.
}

\end{figure}

Current examples of literate programs however feign that writing a
literate program from scratch is a valid programming method; they
do not properly emphasize the importance of formalized system
specification and design. However the descriptive features that
are inherent to the paradigm provide a strong bridge between
specification, design and implementation. Thus it seems possible
to integrate the literate paradigm much further into the software
life cycle. This issue is a field of further research at the TH Darmstadt.

\index{Darmstadt}

\section{``Literate Software Management''}

Software systems today run during their evolution through what is
called a {\em software life cycle}. One iteration of this life cycle
begins with the global aspects of the specification, goes through the
system design and the implementation, runs through the testing phase
and ends with the maintenace of the software system. As todays
software continuously has to react to market or other changes, it is
usual that such systems run through the software life cycle more than
once.

Every phase of the life cycle has it's own tools; most apparent are
programming languages if you think of the implementation phase.
However there also exist tools that help pipeline software systems
through their life cycle, help to keep track of revisions, describe
the overall structure of the system and so on.  The comination of such
tools is normally called a {\em software management system.}

\index{Software management systems}

As mentioned in the end of the previous section, the literate programming
paradigm applies only to small parts of the software life cycle, namely
those that are covered by the term {\em programming in the small\/}.
However the literate paradigm also offers a lot of features that also seem
usefull to the other parts of the software life cycle. The main features
are as follows:
 %
\begin{itemize}

\item Formal and informal description methods are combined leading to
more readable text (the method of ``saying everything twice'').

\item Through the recording of subsequent changes during a phase and
the description of alternatives in the informal text, design
decisions become more transparent and understandable.

\item Additionally, information retrieval methods like index and
cross reference may be a model for similar things on a larger scale.

\end{itemize}

Research is taking place at the TH Darmstadt to apply the benefits of
literate programming to the whole software life cycle. This research
runs under the title ``Software als Buch. Methoden und Werkzeuge f\"ur
das literarische Softwaremanagement'' \cite{Detig92} (what may be
translated as ``Software as a Book. Methods and tools for literate
software management''). A software management system is under
development that incorporates ``literate'' features and shifts the
whole management process to become literature itself; the software
system becomes a software book, or better: a SoftBook.

The underlying model is based upon division of a so called {\em
depot} and a {\em workspace}. The workspace supplies an environment
of working on the software system no matter in what stage of the life
cycle  it should be. The depot provides a general (non-standard)
database that stores documents and system parts from all phases of
the life cycle. These parts can therefore vary a lot: whilst system
parts from the design phase may be written in a formal and abstract
design language, other parts can be simple essays and others may
consist of hard-core assembly code. All these parts are connected
with each other as a hypertext.



\section{The Role of Pretzel}

In the so called workspace, the user may work on the software system
in various ways: he might change the design, implement modules or
just examine  documents that belong to previous versions of the
system. In all these cases the information that is stored in the
depot has to be visualized on the screen. Thinking of the software
system as a book, the output for the user hast to be of high quality
and formal code should be formatted  to local standards. Considering
the large amount of  possible languages that you may encounter in
software development, you may not use a universal module for
formatting. This is where the \pretzel\ program plays its role.

As shown above, it is possible to generate prettyprinting functions
with \pretzel\ for a large variety of languages in fairly short time.
These prettyprinting functions then may be turned into modules of the
software management system that take over some of the ``weaving'' that
is required to visualize the contents of the depot.

Anyway, with \pretzel\ it could be possible to build one's personal
\WEAVE\ system rather easily, because the prettyprinting is the most
intricate part of the weaving process. The only major thing left is to
generate an index and the cross reference and to build the major
frame. A simple approach would be to change the one existing \WEB\
generator, namely the \t{SpiderWEB} system, and change it to use a
\pretzel\ generated prettyprinter.  Another possible way would be to
build a prettyprinting back end for \t{noweb} that is language
independent and uses the \UNIX\ toolbox paradigm.

Prettyprinting will surely remain a major aspect of literate programming
even though a lot of people do not believe in its worth. But, as
Marc van Leeuwen puts it, it is a question of personal taste:

\index{leeuwen@Leeuwen, Marc van}

\begin{quote}

  ``Nevertheless I strongly disagree with people saying that
  pretty-printing is not worth the effort for anyone in general; this
  depends very much on ones particular situation and purpose. In some
  cases pretty-printing may even be the main reason to opt for
  literate programming.'' \cite{vanLeeuwen95}

\end{quote}
