WebWeb ver 2.6




Prof. Dr. Rüdiger Loos, Uwe Kreppel


Arbeitsbereich Computeralgebra

Fakultät für Informatik

Universität Tübingen

Juni 1998


Abstract


A literate programming tool combines code and documentation within a single document. Up to now these tools provided a very inflexible layout based on a specific documentation language. WebWeb gives control over the layout and is a generic approach to documentation language independence. It is based on the features offered by nuweb and implemented in C++ making strong use of the STL.
This is ver 2.6 of WebWeb. It now offers default formatting strings not only for HTML but also for LaTeX and provides a more flexible mechanism for overwriting this default formatting strings.
In addition to HTML CSS is now enabled to provide a very comfortable way of changing the basic layout.




Contents

1 Indroduction
1.1 Why literate programming at all?
1.2 Why WebWeb?
1.3 What's new to former releases?

2 Writing WebWeb
2.1 The Major Commands
2.1.1 Scraps
2.1.2 Flags

2.2 The Minor Commands
2.2.1 Tags Used Outside a Scrap
2.2.2 Tags Used Inside a Scrap

3 Running WebWeb
3.1 Command Line Options
3.2 Restrictions

4 Basic Concepts
4.1 The Standard Template Library - STL

5 The Implementation
5.1 The Overall Structure
5.2 How the STL is used
5.3 The Implementation of the Language Independence
5.3.1 The formatting strings

6 The Program in More Detail
6.1 The main() Function
6.1.1 Errors and Warnings
6.2 The First Pass Over the Source
6.2.1 Parsing the source
6.2.2 Scrap definitions @d, @o, @h, @c
6.2.3 Beginning the Scrap Body - @{
6.2.4 Inside the Scrap Body - @<, @>, @$
6.2.5 Ending the Scrap Body - @}, @+
6.2.6 Setting the Documentation Language Depending Strings
6.2.7 Switching Between docmode and scrapmode
6.2.8 Including Files
6.2.9 Other Tags


6.3 Extracting the Output Files
6.4 Writing the File
6.5 Generating the Documentation
6.5.1 How the generation of the documentation works
6.5.2 Doing the at-line-specific tasks
6.6 Other Functions Needed to Generate the Documentation
6.6.1 formatted_list()
6.6.2 print_formatted_list()
6.6.3 def_list() and ref_lists()
6.6.4 macro_num_list()
6.6.5 header_line()
6.6.6 print_line()
7 Compiling
7.1 The Preprocessor Directives

8 The User's Guide

Appendix A: List of Files

Appendix B: List of Macros

Appendix C: Differences between nuweb and WebWeb

Appendix D: Example

Appendix E: CSS - The STYLE-file

Appendix F: References





1 Indroduction

The basic idea of literate programming [7] is that a programmer writes a document, the web file, combining documentation with code.
This was first introduced in 1984 by Donald E. Knuth [8] when he produced a new language, WEB, which allowed to combine Pascal code with TeX documentation. Therefore, he wrote two tools, weave and tangle, which are run on the web file. tangle aimed at producing a complete Pascal program, ready for compilation.
weave was responsible for the generation of the documentation file which then could be used as input to TeX.


1.1 Why literate programming at all?

 The need for literate programming results from the fact that a well documented program uses a lot of comments in the source code. If it was possible to combine the code with comments that can use the facilities of a documentation language as HTML or even a typesetting language as LaTeX, less work would have to be done when producing a readable documentation that describes the code and the ideas behind it in detail, because it will be generated when the code is created. And whenever changes to the code are made, they can easily be reported in the documentation. So, it can be guaranteed that the documentation is always up to date.

To ensure that a documentation can accompany the software development in a single file it must be possible for the code elements to be placed in any desired order. This allows to present and produce the program top-down, bottom-up or in any way in between. And sometimes it produces a naturally way of modularization of the code.
Together with all kinds of well formulated documentation accompanying the development of a program literate programming has the benefit of being better structured in most of the cases and offering a good base for support and further development.
To distribute a program that was built with a literate programming tool one has only to distribute the web file. The person who gets the web file must simply run the literate programming tool on this file to get all the files holding the code and the documentation. In this case literate programming tools can also be seen as kind of a archive tool creating the archive while creating the program.
The detailed documentation of the code divides it into small units most of them only a few lines long. We will call these pieces scraps from now on.
Every scrap in the web file gets a number and scraps belonging together normally are given a typical name. With this information an index list of all used scraps and cross references between them are created resulting in a comfortable reading of the documentation. The reason is that one can read it from the beginning to the end and can use the same document as a reference to the modules and sometimes even to some single lines of code.
Literate programming became so popular that some more tools were built, for example:
CWEB[6], noweb [10], , nuweb [2], , SchemeWEB [9] , ...
Basically literate programming tools can differ from each other in two points:
First, the support of a programming language and second, the documentation language.
The documentation language is TeX/LaTeX for all of the tools mentioned above. The supported programming language is C/C++ for CWEB and any dialect of Lisp for SchemeWEB. But noweb and nuweb support no programming language at all. Due to this fact, they can be used to create code in any programming language. To create code from the web file noweb and nuweb just bring the scraps in the right order by replacing some macros that can be specified in the webfile and copy these scraps into the appropriate output files. We will see later on how this is done.
Now one can ask: 'Why should someone build another literate programming tool', or:


1.2 Why WebWeb?

 As noweb and nuweb are independent from the programming language, we make an approach to the independence of the documentation language. This independence is based on two decisions concerning the design:
First we attach different modules to the language dependent and the language independent parts of the formatter respectively. Second, we provide a slightly generic mechanism to overwrite the default behavior of the module responsible for the language depending tasks.

With WebWeb it is not only possible to adjust the default behavior of the formatting to the users needs, but also to choose another documentation language by just giving the formatting strings.
So far default settings for HTML and LaTeX are provided, because in our opinion these two languages are used most for the documentation of programm code. [11], .

Aside from this new feature we implemented some other extensions to the functionality of nuweb which we have used before we developed WebWeb.
We will just mention them very briefly, because they are listed in Appendix C in more detail:



1.3 What's new to former releases?

Because WebWeb's performance was very poor we changed the function paramaters to a const reference type whenever possible. So we can avoid the time consuming copying of strings and even whole containers.



2 Writing WebWeb

 The major part of a WebWeb file will be ordinary documentation written in the documentation language, say HTML. In fact, any HTML file can serve as input to WebWeb and will be simply copied through without changes to the documentation file - unless a WebWeb command is discovered. All WebWeb commands begin with a @ (at-sign). Therefore, a file without at-signs will be copied unchanged. WebWeb commands are used to specify output files, define macros, and delimit scraps. These are the basic features of interest to the WebWeb tool - the remaining simply consists of text to be copied to the documentation file.
Throughout this document a WebWeb command is sometimes simply referd to as a tag.


2.1 The Major Commands

 Files and macros are defined by the following commands:
@o file-name flags scrap :
Output a file. The file name is terminated by whitespace.
@d macro-name scrap :
Define a macro. The macro name is terminated by a return or the beginning of a scrap.
@h macro-name scrap :
Define a macro as with @d but if invoked in a scrap, it will be only expanded in the output file and not in the documentation. So scraps can be excluded from the documentation. If invoked in normal text of a documentation section, it will be expanded to the documentation. The macro definition itself will never be documented. So it fulfils two tasks: First, it allows to hide part of the code from the documentation and second, it represents the macro mechanism for text in the documentation.
@c macro-name scrap :
The whole macro is commented out. Can be used for comments in the web file that should not be seen in the documentation as well.
A specific file may be specified several times, with each definition being written out, one after the other, in the order they appear. The procedure of defining macros is likewise.

2.1.1 Scraps

 Scraps have a specific begin marker and end marker to allow precise control over the content. Note that any amount of whitespace (including carriage returns) may appear between a name and the beginning of a scrap.
@{ anything @} :
where the scrap body includes every character in anything - all the blanks, all the tabs, all the carriage returns. The scrap will be typeset in verbatim mode.
Inside a scrap, we may invoke a macro.
@< macro-name @>
Causes the macro macro-name to be expanded inline as the code is written out to a file. Recursive macro invocations are recognized.
When a macro call within a scrap is substituted by the macro, the body of the macro is indented to match the indentation of the macro invocation. Therefore, care must be taken with languages (e.g., Fortran) that are sensitive to indentation. These default behaviors may be changed for each output file (see below).

2.1.2 Flags

 When defining an output file, the programmer has the option of using flags to control the output of a particular file. They introduce little language dependences; however, they do so only for a particular file. Thus, it is still easy to mix languages within a single document. WebWeb provides two 'per'-file flags:
-l :
Forces the creation of #line directives in the output file. These are useful with C (and sometimes C++ and Fortran) since they cause the compiler's error messages to refer to the web file rather than the output file. Similarly, they allow source debugging in terms of the web file.
-i :
Suppresses the indentation of macros. That is, when a macro is expanded in a scrap, it will not be indented to match the indentation of the macro invocation. This flag would seem most useful for Fortran programmers.



2.2 The Minor Commands

 We have a set of utility commands. Most of them may appear not throughout the whole document but only in specific parts, either inside a scrap or outside.

2.2.1 Tags Used Outside a Scrap
@@
causes a single at-sign to be printed.
@l language
WebWeb is designed in a way that it can hold for every common documentation language a set of strings that perform the default formatting. To choose between these languages we provide this tag. Until now (ver 2.6), a default behavior is included for HTML and LaTeX.
@i filename
includes the file filename. Nested loops are recognized and WebWeb terminates with an error message.
@f
prints list of all output files that are defined with @o.
@m
prints list of macros.
@u
prints the user defined index that is generated with @+. it is alphabetically sorted.
@| text @|
text will be displayed in the same font as the scraps.
@<scrapname@>
macro call to scrapname. Scrap scrapname will be expanded here. Note that the scrap to be expanded must be defined as a hidden scrap using @h.

2.2.2 Tags Used Inside a Scrap
@@
causes a single at-sign to be printed.
@<scrapname@>
macro call to scrapname. Scrap scrapname will be copied to the output file.
@$ text @$
text will be displayed in the same font as the documentation sections.
@+ string
puts string in the user defined index. Must be followed by another @+ or by @} at the end of a scrap.
Each string that appeares in a scrap and should be added to the user defined index has to be specified at the end of a scrap.
@}
Used to end the body of a scrap.




3 Running WebWeb

 WebWeb is invoked using the following command:
webweb flags file-name
The file name must have the extension .ww
While a file name may specify a file in another directory, the resulting documentation file will always be created in the current directory. Its extension is by default .html. For every included 'default' language the right extension is set. The name of the documentation file can also be given as an option.



3.1 Command Line Options

 By default, WebWeb performs both tangling and weaving. It is possible to achieve faster throughput by avoiding one or another of the default functions using command-line flags. There are currently three possible flags:
-d
Suppress generation of the documentation file.
-o
Suppress generation of the output files.
-f
Forces the overwriting of the output files without testing for change.
Thus, the command
webweb -d -o ernie.ww
would simply scan the input and produce no output at all.
There are two additional command-line flags:
-i
interactive mode. Asks for each output file as well as for the documentation file whether to write it or not.
-N doc-file-name
Writes the documentation to the file doc-file-name.
So the usage of WebWeb is:

webweb [-h] [-help] [-i] [-f] [-d] [-o] [-s] [-N docfileName] filename
An example of a web file and the documentation produced by WebWeb is shown in Appendix D.


3.2 Restrictions

 Currently there are some restrictions. We will delete them in further development.
These restrictions are:





4 Basic Concepts

 Two basic ideas lead to this design:

First, we wanted an approach to the independence of the documentation language, because we saw that the support of LaTeX as done by nuweb is not enough in times when even whole books are written directly in HTML. It is true that nuweb, for example, provides an option to produce an output that can be reinterpreted by the tool latex2html [3] and get the desired HTML documentation, but we wanted even more. We aimed at a literate programming tool that 'supports' on the one hand forthcoming versions of existing documentation languages and, on the other hand, the possibility of using a lot of different documentation languages in order to enable the user to chose his favorite.
Moreover we intended to simplify the use of literate programming as there should be no need to learn a documentation language that is so non-trivial as LaTeX ( although the author has the opinion that once having learned LaTeX you will see that it looks perfectly as typeset as you want and if you have to write many mathematical terms you will be gratefully using this language).

This was the guiding idea of the whole project. As nuweb and stweb (a further development of nuweb) provided the independence of programming language by strictly copying the scraps in the output files after they were put in the right order, WebWeb should provide language independence in addition. This made a mechanism for cross referencing absolutely vital, because the one provided by the documentation languages (if at all) is not uniform. To do so we adapted a technique from a former work of the author that is based on associative arrays.

We chose to implement WebWeb in C++ [5], because of three advantages:
  1. C++ compiler are nearly available on each system
  2. the C++ compiler produce fast executables
  3. C++ comprises the new STL
This brought us the second idea for our new tool:
We wanted to make an extensive use of the STL, if possible without creating a single class but only using the datatypes and classes provided by the STL and then see if and how it works. But first a few words to the STL [1], [4], [5] itself:


4.1 The Standard Template Library - STL

 At the end of 1994 the STL as a main part of the C++ standard library was integrated in the C++ standardization process which started in 1989. The STL heavily depends on the template mechanism. In general this gives the opportunity to create classes with a parametrization of the datatypes used in the class. So, the STL sets up a new level of abstraction.
We find a set of well structured generic C++ components working fast and comfortably when combined with each other or with user defined components and algorithms.

The three main kinds of components that are used in WebWeb are:
Container, iterator, algorithm.

Containers are used to deal with a set of objects of a defined type. There exist different types of containers according to the special needs of the user. They can be classified in sequential (vector, deque, list) and associative (set, multiset, map, multimap) containers.
Iterators are objects, which iterate over the container. Each iterator object represents a position in the container. Some basic operators as iterator::operator++(), iterator::operator==() and iterator::operator*() define the behavior of the iterator. These iterators can not only be used with all container but also with arrays and strings.
Algorithms are used to work at the elements of a container. There are algorithms for swapping, sorting, copying, adding and modifying elements and/or containers. These algorithms are not memberfunctions of the containers but are globally defined to work with the iterators. So, they had to be implemented only once and can easily be replaced by proprietary algorithms. Moreover they can be used for self created classes.


This separation of data from the functions working on it, however, is not object oriented. Nevertheless, it prevents the library from getting too huge and makes it very mighty.
For further information on the STL see the references.



5 The Implementation

 As stated above, WebWeb is implemented in C++. At the first stage of our project, we used the literate programming tool stweb in order to create the basic features of WebWeb. Thereafter, we could use WebWeb to generate itself.
This procedure allows for using newly created features for further development of WebWeb. Let us illustrate this taking a look at hidden scraps:
I did not want the user's guide to appear in the file webweb.html because the user's guide is thought of as a kind of tabulated reference only listed in the special file wwguide.html. Therefore, I needed a kind of hidden scrap that had to be implemented. Until the implementation was carried out, the user's guide was defined with @d as a 'normal' scrap and therefore appeared in the file webweb.html. After the creation of the hidden scrap I simply had to replace the @d with @h. Now the users's guide was invisible in the documentation of WebWeb but the file wwguide.html was still created.

5.1 The Overall Structure

 Files and macros can be written in small units that are concatenated in the order of occurrence in the web file into a single scrap for each file and macro.
In the same order they are numbered in the HTML-documentation.

The source is read and each chunk of text is stored in a STL-Container. In this first pass, all information needed for making the cross-references, the output-files etc, will also be stored in appropriate STL-containers. This gives the overall design of the program:

« webweb.cpp » 1

#include "webweb.h"
<functions and routines: 11, 32, 33, 52, 64, 65, 66, 69>
<read the input files: 18>
<make the files: 51>
<the main function: 9>

We maintain a header file. All global declarations, typedefs and preprocessor directives are put there.

« webweb.h » 2

<preprocessor directives: 96>
<some typedefs: 3>
<global declarations: 4, 5, 6, 7, 12, 15, 16, 19, 21, 26, 29, 30, 37, 39, 44, 53, 55, 63, 71, 73>
<hidden declarations: 27, 34, 60>




5.2 How the STL Is Used

 We work with five sorts of STL-Containers and use typedefs to define them.

« some typedefs » 3

typedef list< int >           text_t;
typedef map< string, text_t > map_t;
typedef vector< string >      buf_t;
typedef map< string, string > opt_t;
typedef vector< int >         line_t;
enum doc_t { NONE, HTML, LATEX, WfW }; 
doc_t docLang = NONE;

Scrap referenced in: 2

For each kind of scrap we maintain a STL-container. This seems adequate as we shall see when examine the creation of the documentation and the output files. A fourth container is used for the user-made index.

« global declarations » 4

map_t            scrap;
map_t            files;
map_t            hide;
map_t            merged;

Also defined in: , 5 ...

Scrap referenced in: 2




5.3 The Implementation of the Language Independence

 Steps taken in this context:

5.3.1 The formatting strings

 We have two aims: First, to provide the appropriate formatting strings for the common documentation languages as HTML and LaTeX, and second, to create the possibility of overwriting these strings respectively to generate the appropriate strings for working in another documentation language.
To achieve this, we have seventeen formatting strings and two strings defining special characters. This technique allows to provide default settings for each documentation language wanted.
To get an insight in this technique we present the formatting strings for the two default languages, LaTeX and HTML.
First, we list the default settings to support LaTeX.

« global declarations » 5

string     begSStrLATEX   = "{\\vspace*{-0.5ex} \\begin{flushleft}";
string     endSStrLATEX   = "\\end{flushleft} }";
//string     headerStringLATEX = "{\\bf $\\langle$ scrapname, \\textit{scrapnumber} $\\rangle$ }";
string     headerStringLATEX = "{{\\bf \\em scrapname}, \\textit{scrapnumber}\\\\[-0.5em] \\rule{7cm}{0.5pt}\\\\}";
string     defStringLATEX    = "{\\small \\it Also defined in: list of numbers }\\\\";
string     refStringLATEX    = "{\\small \\it Referenced in: list of numbers }\\\\";
string     numStringLATEX      = " scrapnumber";
string     numSeparatorLATEX      = ",";
//string     bodyStartStringLATEX   = "\\vspace*{-2ex} \\begin{quote}";
//string     bodyStartStringLATEX   = "\\vspace*{-2ex} {\\small";
string     bodyStartStringLATEX   = "{\\footnotesize";
//string     bodyEndStringLATEX     = "\\end{quote} \\vspace*{-6ex} $\\diamond$ \\newline"; 
//string     bodyEndStringLATEX     = "} \\vspace*{-2ex} $\\diamond$ \\vspace*{2ex} \\newline"; 
string     bodyEndStringLATEX     = "} \\vspace*{-3ex} \\newline \\rule{7cm}{0.5pt}\\\\"; 
//string     bodyEndStringLATEX     = "} \\vspace*{-2ex} \\newline"; 
string     macroCallNumLATEX     = "} : {\\small list of numbers }";
string     startMacroCallLATEX   = "@ $\\langle$ {\\bf ";
string     endMacroCallLATEX    = "$\\rangle$ \\verb@";

string     toDocmodeStringLATEX   = "@";
string     toScrapmodeStringLATEX   = "\\verb@";

string listStartFormatLATEX = "\\begin{description}";
string listEndFormatLATEX = "\\vspace*{2ex} \\end{description}";
string listEntryFormatLATEX = "\\item[scrapname: ] list of numbers \\vspace{-2ex}";

string     ltLATEX         = "$\\langle$";
string     gtLATEX         = "$\\rangle$";

Also defined in: ... 4, 6 ...

Scrap referenced in: 2

Now, we list the appropriate settings used to format a documentation that is written in HTML. Note, that we make use of the comfortable Casscading Style Sheets.

« global declarations » 6

string     begSStrHTML = "<P CLASS=\"clsScrpStrt\"> </P>";
string     endSStrHTML = "<P CLASS=\"clsScrpEnd\"> </P>";
string     headerStringHTML = "<H4 CLASS=\"clsScrpHd\"><A CLASS=\"clsScrpHdA\" NAME=\"scrap scrapnumber\">« scrapname » scrapnumber</A></H4>";

string     defStringHTML    = "<P CLASS=\"clsScrpDf\">Also defined in: list of numbers </P>";
string     refStringHTML    = "<P CLASS=\"clsScrpRf\">Scrap referenced in: list of numbers </P>";
string     numStringHTML      = " <A CLASS=\"clsScrpNumHrf\" HREF=\"#scrap scrapnumber\">scrapnumber</A>";
string     numSeparatorHTML      = ",";
string     bodyStartStringHTML   = "";
string     bodyEndStringHTML     = "</PRE><BR>"; 
string     toScrapmodeStringHTML   = "<PRE CLASS=\"clsScrpMd\">";
string     toDocmodeStringHTML     = "</PRE COLOR=\"Black\">"; 
string     macroCallNumHTML     = ":</CODE>list of numbers";
string     startMacroCallHTML   = "<<CODE CLASS=\"clsMcrCll\">";
string     endMacroCallHTML    = ">";

string listStartFormatHTML = "<TABLE CLASS=\"clsTbl\">";
string listEndFormatHTML = "</TABLE>";
string listEntryFormatHTML = "<TR><TD CLASS=\"clsTblNm\">scrapname : <TD CLASS=\"clsTblEntry\">list of numbers";

string     ltHTML         = "<";
string     gtHTML         = ">";

Also defined in: ... 5, 7 ...

Scrap referenced in: 2

If these settings are not overwritten (which is indicated by the boolean variable default_setting ) in the web file by the tags @sc, (with c a character as listed in the user's guide) they are mapped to the current formatstrings i.e. we work with these strings when generating the documentation.

« global declarations » 7

string     metaName   = "scrapname";
string     metaNumber = "scrapnumber";
string     metaList   = "list of numbers";
string     lt         = "<";
string     gt         = ">";

string     empty_str ="";

string*     begSString;
string*     endSString;
string*     headerString;
string*     defString;
string*     refString;
string*     numString;
string*     numSeparator;
string*     bodyStartString;
string*     bodyEndString;
string*     macroCallNum;
string*     startMacroCall;
string*     endMacroCall;
string*     toDocmodeString;
string*     toScrapmodeString;

string*     listStartFormat;
string*     listEndFormat;
string*     listEntryFormat;
bool       default_setting = 1;

Also defined in: ... 6, 12 ...

Scrap referenced in: 2

It goes without saying that the right choice for the default settings depends on the 'default' documentation language that is used.

« init_formatting_strings(): part of language independence » 8

void init_formatting_strings() {
  if (wwdebug) wwdb("init_formatting_strings");
  switch ( docLang ) {
    case HTML: { 
      if ( default_name )
        docfile          = basename + ".html";
      if ( ! begSString)
        begSString     = &begSStrHTML;
      if ( ! endSString)
        endSString     = &endSStrHTML;
      if ( ! headerString)
        headerString     = &headerStringHTML;
      if ( ! defString )
        defString        = &defStringHTML;
      if ( ! refString )
        refString        = &refStringHTML;
      if ( ! numString )
        numString        = &numStringHTML;
      if ( ! numSeparator )
        numSeparator        = &numSeparatorHTML;
      if ( ! bodyStartString )
        bodyStartString  = &bodyStartStringHTML;
      if ( ! bodyEndString )
        bodyEndString    = &bodyEndStringHTML;
      if ( ! toDocmodeString )
        toDocmodeString  = &toDocmodeStringHTML;
      if ( ! toScrapmodeString )
        toScrapmodeString  = &toScrapmodeStringHTML;
      if ( ! macroCallNum )
        macroCallNum     = ¯oCallNumHTML;
      if ( ! startMacroCall )
        startMacroCall   = &startMacroCallHTML;
      if ( ! endMacroCall )
        endMacroCall     = &endMacroCallHTML;
      if ( ! listStartFormat )
        listStartFormat  = &listStartFormatHTML;
      if ( ! listEndFormat )
        listEndFormat    = &listEndFormatHTML;
      if ( ! listEntryFormat )
        listEntryFormat  = &listEntryFormatHTML;

      if ( ltHTML != lt && ltHTML!= "" )
        lt = ltHTML;
      if ( gtHTML != gt && gtHTML!= "" )
        gt = gtHTML;
      break;
    }
    case LATEX : { 
      if ( default_name )
        docfile          = basename + ".tex";
      if ( ! begSString)
        begSString     = &begSStrLATEX;
      if ( ! endSString)
        endSString     = &endSStrLATEX;
      if ( ! headerString)
        headerString     = &headerStringLATEX;
      if ( ! defString )
        defString        = &defStringLATEX;
      if ( ! refString )
        refString        = &refStringLATEX;
      if ( ! numString )
        numString        = &numStringLATEX;
      if ( ! numSeparator )
       numSeparator        = &numSeparatorLATEX;
      if ( ! bodyStartString )
        bodyStartString  = &bodyStartStringLATEX;
      if ( ! bodyEndString )
        bodyEndString    = &bodyEndStringLATEX;
      if ( ! toDocmodeString )
        toDocmodeString  = &toDocmodeStringLATEX;
      if ( ! toScrapmodeString )
        toScrapmodeString  = &toScrapmodeStringLATEX;
      if ( ! macroCallNum )
        macroCallNum     = ¯oCallNumLATEX;
      if ( ! startMacroCall )
        startMacroCall   = &startMacroCallLATEX;
      if ( ! endMacroCall )
        endMacroCall     = &endMacroCallLATEX;
      if ( ! listStartFormat )
        listStartFormat  = &listStartFormatLATEX;
      if ( ! listEndFormat )
        listEndFormat    = &listEndFormatLATEX;
      if ( ! listEntryFormat )
        listEntryFormat  = &listEntryFormatLATEX;
      if ( ltLATEX != lt && ltLATEX!= "" )
        lt               = ltLATEX;
      if ( gtLATEX != gt && gtLATEX!= "" )
        gt               = gtLATEX;
      break;
    }
    case NONE:
    default: {
      docfile = basename + ".txt";
      if ( ! begSString)
        begSString     = &empty_str;
      if ( ! endSString)
        endSString     = &empty_str;
      if ( ! headerString)
        headerString     = &empty_str;
      if ( ! defString )
        defString        = &empty_str;
      if ( ! refString )
        refString        = &empty_str;
      if ( ! numString )
        numString        = &empty_str;
      if ( ! numSeparator )
       numSeparator        = &empty_str;
      if ( ! bodyStartString )
        bodyStartString  = &empty_str;
      if ( ! bodyEndString )
        bodyEndString    = &empty_str;
      if ( ! toDocmodeString )
        toDocmodeString  = &empty_str;
      if ( ! toScrapmodeString )
        toScrapmodeString  = &empty_str;
      if ( ! macroCallNum )
        macroCallNum     = &empty_str;
      if ( ! startMacroCall )
        startMacroCall   = &empty_str;
      if ( ! endMacroCall )
        endMacroCall     = &empty_str;
      if ( ! listStartFormat )
        listStartFormat  = &empty_str;
      if ( ! listEndFormat )
        listEndFormat    = &empty_str;
      if ( ! listEntryFormat )
        listEntryFormat  = &empty_str;
      break;
    }
  }

  if ( ! begSStrU.empty() ) begSString  = &begSStrU;
  if ( ! endSStrU.empty() ) endSString  = &endSStrU;
  if ( ! headStrU.empty() ) headerString  = &headStrU;
  if ( ! defStrU.empty() ) defString  = &defStrU;
  if ( ! refStrU.empty() ) refString  = &refStrU;
  if ( ! numStrU.empty() ) numString  = &numStrU;
  if ( ! numSepU.empty() ) numSeparator  = &numSepU;
  if ( ! begBStrU.empty() ) bodyStartString  = &begBStrU;
  if ( ! endBStrU.empty() ) bodyEndString  = &endBStrU;
  if ( ! macroNumStrU.empty() ) macroCallNum  = ¯oNumStrU;
  if ( ! begMStrU.empty() ) startMacroCall  = &begMStrU;
  if ( ! endMStrU.empty() ) endMacroCall  = &endMStrU;
  if ( ! begLStrU.empty() ) listStartFormat  = &begLStrU;
  if ( ! endLStrU.empty() ) listEndFormat  = &endLStrU;
  if ( ! listStrU.empty() ) listEntryFormat  = &listStrU;

}

Scrap referenced in: 69





6 The Program in More Detail

 Now we take a closer look at the code. We first inspect the main() function which makes the first and only pass over the web file itself. Then, we will examine how the generation of the output files is done and finally look at how the documentation is produced.


6.1 The main() Function

 First we set the variable basename to the basename of the web file. Then, we evaluate the given command options. After that, we call three routines named according to their basic tasks:

« the main function » 9

int main(int argc, char * argv[]) {
<evaluate arguments: 10, 13, 14, 17>
read_source_file(argv[si]);
if (mkdoc)
  make_doc();
if (mkout)
  make_files();
return 0;
}

Scrap referenced in: 1

We assume that the name of the source file is the last given argument. So, at least one argument must be found. If not, WebWeb will exit sending an error message.

« evaluate arguments » 10

  if (argc == 1) error("Usage: " + string(argv[0]) + " file", 0);

Also defined in: , 13 ...

Scrap referenced in: 9



6.1.1 Errors and Warnings

 We use the first occurence of an error message as shown above to demonstrate how error messages and warnings are treated.

Errors are sent by the function error() taking a string as argument. It copies the string to cerr and exits with code 1.
We also provide an adequate warning function which copies the given string to cout.
Error and warning functions need a second argument: An integer that should refer to the number of the current at_line because then the real line number of the source file will be computed and copied either to cerr or to cout.

« functions and routines » 11

void error(const string& s1, const int& part_no) {
  if ( part_no == 0 ) {
    cerr << s1;            
    exit(1);
  }
  if ( lines.empty() ) 
    compute_line_numbers();

  cerr << "(" << actual_line_number() << "):  " << s1 << endl;
  exit(1);
}
 
void warning(const string& s1, const int& part_no) {
  if ( part_no == 0 )
    cerr << "WARNING:  " << s1 << endl;            
  else {
    if ( lines.empty() ) 
      compute_line_numbers() ;
    cerr << "WARNING (" << actual_line_number() << "):  " << s1 << endl;
  }
}
 
void wwdb(const string& s, string s2 = "") {
  cerr << "Funktion: \"" << s << s2 << "\"" << endl;
}

Also defined in: , 32 ...

Scrap referenced in: 1

The name of the given file name is stored in the variable source which is of type string. If the source file name has the extension .ww, the basename will be stored in basename. Both are defined in the header file.

« global declarations » 12

string           basename;
string           source;

Also defined in: ... 7, 15 ...

Scrap referenced in: 2

As most software does we provide an option to print out a commented list of the command line options.

« evaluate arguments » 13

version = "This is WebWeb,  ";
helpstr = version;
helpstr += "\n \nThe command line options are:\n"; 
helpstr += "  [-h] [-help]      This text.\n";
helpstr += "  [-d]              Generate NO documentation but extract the files.\n";
helpstr += "  [-o]              Generate NO output files but make the documentation.\n";
helpstr += "  [-i]              For each file to create, check whether it already\n";
helpstr += "                      exists and if so, ask the user if WebWeb should\n";
helpstr += "                      overwrite it. (interactive)\n";
helpstr += "  [-f]              Force the overwriting of already existing files.\n";
helpstr += "  [-s]              Do not print the version when starting WebWeb. (silent)\n";
helpstr += "  [-N fname]        Write the documentation to file \'fname'\n";

Also defined in: ... 10, 14 ...

Scrap referenced in: 9

The basename is needed to construct the name of the documentation file. If no source file is given, WebWeb exits with an error message.

« evaluate arguments » 14

int si = argc - 1;
source = argv[si];
  if ( source == "-h" ) {
    cout << helpstr << endl;
    exit(1);
  }
  if ( source == "-help" ) {
    cout << helpstr << endl;
    exit(1);
  }
size_t ppos = source.find('.');
if ( ppos == string::npos ) {
  basename = source;
  source = source + ".ww";
}
else {
  if ( source.substr(ppos) != ".ww" ) 
    error(" Wrong extension to filename. Must be  <" + source + ".ww>", 0);
  basename = source.substr(0, ppos);
}

Also defined in: ... 13, 17 ...

Scrap referenced in: 9

Then, we evaluate the arguments given after the command name. Therefore, we loop over the given options and set appropriate variables which are mostly of type bool and set them in the header file to their default values.

« global declarations » 15

bool wwdebug = 0;
bool force = 0;
bool interactive = 0;
bool mkdoc = 1;
bool mkout = 1;
bool default_name = 1;
bool css = 1;
string    docfile;

Also defined in: ... 12, 16 ...

Scrap referenced in: 2

If an invalid option is found, WebWeb will terminate and copy the string usage to error();

« global declarations » 16

string           usage;
string           version;
string           helpstr;

Also defined in: ... 15, 19 ...

Scrap referenced in: 2

« evaluate arguments » 17

 
bool silent = 0;
string argument;
usage = "USAGE:  webweb [-h] [-help] [-o] [-d] [-i] [-f] [-s] [-N docfileName] sourcefile";
for ( int counter = 1; counter < si; counter++) {
  argument = argv[counter];
  if ( argument == "-f" ) {
      force = 1;
      continue;
  }
  if ( argument == "-i" ) {
      interactive = 1;
      continue;
  }
  if ( argument == "-o" ) {
      mkout = 0;
      continue;
  }
  if ( argument == "-d" ) {
      mkdoc = 0;
      continue;
  }
  if ( argument == "-s" ) {
      silent = 1;
      continue;
  }
  if ( argument == "-debug" ) {
      wwdebug = 1;
      continue;
  }
  if ( argument == "-N" ) {
      docfile = argv[++counter];
      default_name = 0;
      continue;
  }
  error("Wrong argument: " + argument + "\n" + usage, 0);           
}
if ( ! silent )
cout << version << endl;

Also defined in: ... 14

Scrap referenced in: 9




6.2 The First Pass Over the Source

 As the description of the structure of the main() function has shown, we read the source by calling the function read_source_file(). For this purpose, we make use of the comfortable class ifstream which is a realization of the template class basic_ifstream. It takes the name of the web file as an argument and tries to open it. Then, we set the local variables include_file and incl_filename. We will turn to the meaning of this variables later on.

« read the input files » 18

void read_source_file(const char_t* fileName) { 
  if (wwdebug) wwdb("read_ource_file");
  state = doc;
  ifstream is(fileName);
  if ( ! is ) warning("Cannot open file " + string(fileName), 0);
//  istream::pos_type firstPos = is.tellg();
//  is.seekg(-1, ios::end);
//  istream::pos_type lastPos = is.tellg();
//  is.seekg(0, ios::beg);
//  if ( (lastPos - firstPos) >= 10000 )
//    bufferSize = (lastPos - firstPos);
//  else 
//    bufferSize = 10000; 
//cout << "FIRST:  " << firstPos << "LAST:  " << lastPos << endl;
  bool include_file = 0;
  string incl_filename;
  if ( is ) {
    <read the source: 20>
  }
}

Scrap referenced in: 1

« global declarations » 19

  string line, line1;
  int bufferSize;

Also defined in: ... 16, 21 ...

Scrap referenced in: 2

We scan with getline over the web file and set the optional end-of-line character to @ which gives us the partitioning of the source. Then, we take a lookahead on the following character which indicates what to do. We store each read chunk of text in the STL-container buf which is a vector of strings. So, the contents of buf represents nearly the whole source, except for comments and @-signs, part of a tag, which are not copied.

« read the source » 20

<work on first line: 22>
do {
  ++part_no;
  char_t c = tolower(static_cast< char_t >(is.peek()));

  <include a file: 49>
  <substitute double at-sign: 23>
  <skip comments: 24>
  <set documentation language: 42>
  <set formatting strings: 45>

  line = fake + line;
  fake = "";

  buf.push_back(line);
  ++buf_no;

  if ( ! comment ) {
    <analyze the source line: 25>
  }
} 
while ( getline(is, line, at_sign) );

Scrap referenced in: 18

Most variables are declared global.

« global declarations » 21

typedef string::value_type char_t;
const char_t at_sign = '@';
buf_t            buf;
int part_no = -1;
int buf_no = -1;
bool comment = 0;
string fake;

Also defined in: ... 19, 26 ...

Scrap referenced in: 2

Since we assume that the first at_line of each source file given at the command line or with the @i tag belongs to the documentation, we get some at_lines which do not start with a character special to WebWeb. In order to simplify the generation of the documentation we want each at_line to start with a special character. Consequently, we add a fake character, namely the z.

« work on first line » 22

getline(is, line, at_sign); 
line = 'z' + line;

Scrap referenced in: 20

If the lookahead gives us another @ we will call getline twice and store the read characters in line1:
First, we read the lookahead. Second, we read the line until the next @ is found and concatenate this to the characters stored in line. This aims at reducing any double at-sign to a single one. Thus, it is guaranteed that any at_line starts with a valid meta character.

« substitute double at-sign » 23

while ( c == '@' ) {
  getline(is, line1, at_sign);
  getline(is, line1, at_sign);
  line +=  "@" + line1;
  c = tolower(static_cast< char_t >(is.peek()));
}

Scrap referenced in: 20

To skip the comments we use the boolean variable comment. It is set to 1 when is.peek holds a c while scanning the input line.

« skip comments » 24

if ( comment ) {
  if ( c == '}' ) comment = 0;
  continue;
}

Scrap referenced in: 20


6.2.1 Parsing the source

 To parse the source we take a look at the lookahead and on the state.
For two reasons, the lookahead can be invalid:
First, if it is not part of a tag and second, if it occurs in the wrong state.

« analyze the source line » 25

switch ( c ) {
  <switch the lookahead: 28, 31, 35, 36, 38, 40, 41, 43, 46, 47, 48, 50>
  default : {
    if (  is.peek() != EOF ) 
      error("Unexpected  @. Perhaps you forgot to escape it?", buf_no );
    break; 
  }
}

Scrap referenced in: 20

So we run a finite automaton through the setting of the variable state. This happens in dependence on the lookahead which can be viewed as the input token from the scanner

« global declarations » 26

enum state_t { doc, header, body, macro, scrapmode, docmode, uindex, docdef, fdef };
state_t state = doc;

Also defined in: ... 21, 29 ...

Scrap referenced in: 2

« hidden declarations » 27

bool isfile = 0;
bool hidden = 0;

Also defined in: , 34 ...

Scrap referenced in: 2


6.2.2 Scrap definitions @d, @o, @h, @c

 Let us first take a look at the different ways of defining a scrap.

« switch the lookahead » 28

  case 'd': {
    if ( state == doc ) {
      state = header;
      map_ptr = &scrap;
    }
    else
      error("You cannot start a scrap definition here.", buf_no);
    break;
  }
  case 'o': {
    if ( state == doc ) {
      state = header;
      isfile = 1;
      map_ptr = &files;
    }
    else
      error("You cannot start a file definition here.", buf_no);
    break;
  }
  case 'h': {
    if ( state == doc ) {
      state = header;
      map_ptr = &hide;
      hidden = 1;
    }
    else
      error("You cannot start a hidden scrap definition here.", buf_no);
    break;
  }
  case 'c': {
    comment = 1;
    break;
  }

Also defined in: , 31 ...

Scrap referenced in: 25

As we want to store the references to the following at-line in the appropriate container until the end of the scrap is reached, we maintain a pointer to the map_type. This pointer is always set to the appropriate container.

« global declarations » 29

map_t*           map_ptr;

Also defined in: ... 26, 30 ...

Scrap referenced in: 2


6.2.3 Beginning the Scrap Body - @{

 Since every scrapheader must be followed by a scrapbody, a @{ indicates the end of a header and the beginning of the body. This means, the lookahead is a }. We will set the state to body and remove all leading and trailing spaces.
That is the adequate moment to collect the numbers of all scraps defining the same macro. This is done by pushing back the current scrapnumber to the container def_nums. scrap_name holds the current scrapname and is valid while processing the whole scrap. num simply counts the scraps.

« global declarations » 30

map_t            def_nums;
string           scrap_name;
int              num = 0;
string dL;

Also defined in: ... 29, 37 ...

Scrap referenced in: 2

This will only be the case if the scrap is not of kind hidden.

« switch the lookahead » 31

case '{': {
  if ( state == header ) {
    state = body;
    if ( isfile ) {
      scrap_name = grab_o_name(line);
      options[scrap_name] += opt;
      isfile = 0;
    }
    else
      scrap_name = grab_d_name(line); 
    if ( ! hidden )
      def_nums[ scrap_name ].push_back(++num); 
  }
  else
    error("unexpected @{", buf_no);
    
  break;
}

Also defined in: ... 28, 35 ...

Scrap referenced in: 25

As we see, two functions are called here:
grab_d_name and grab_o_name. Both remove the newlines, spaces, and tabs at the beginning and at the end of each scrapname using the comfortable functions of the string library of the STL:

« functions and routines » 32

string grab_d_name(const string& line) {    
    if (wwdebug) wwdb("grab_d_name: ", line);
    s = line.find_first_not_of(" \t\n", 1);
    e = line.find_last_not_of(" \t\n");
    return line.substr(s, e - s + 1);
}

Also defined in: ... 11, 33 ...

Scrap referenced in: 1

grab_o_name has a slightly broader effect.
It is called when the scrapname is the name of an output file. In this case the name may be followed by some filename options which should not be part of the scrapname. These options are stored in the variable opt. For every filename we provide an entry in the container options which will be evaluated when the output files are written.

« functions and routines » 33

string grab_o_name(const string& line) {
    if (wwdebug) wwdb("grab_o_name", line);
    s = line.find_first_not_of(" \t\n", 1);
    e = line.find_first_of(" \t\n", s);
    opt = line.substr(e);
    return line.substr(s, e - s );
}

Also defined in: ... 32, 52 ...

Scrap referenced in: 1

« hidden declarations » 34

opt_t            options;
string           opt;
size_t s;
size_t e;

Also defined in: ... 27, 60 ...

Scrap referenced in: 2


6.2.4 Inside the Scrap Body - @<, @>, @$

 In the body of a scrap only two special cases result in a particular treatment of the following line
This is described at the end of this chapter. In this case we must first get a < from the lookahead. We then set the state to macro and store the current line index in the appropriate container.

« switch the lookahead » 35

case '<': {                                          
  if ( state == body ) {
    (*map_ptr)[ scrap_name ].push_back(buf_no);
    state = macro;
  }
  if ( state == header )
    error("Do not use macro extension in scrapheader: ", buf_no);
  break;
}

Also defined in: ... 31, 36 ...

Scrap referenced in: 25

In this state the next lookahead must be a '>'. If so, we push back the read line to the appropriate container.

« switch the lookahead » 36

case '>': {
  if ( state == macro ) {
    (*map_ptr)[ scrap_name ].push_back(buf_no);
    state = body;
  }
  else {
    if ( state == body ) 
      error("First invoke a macro call or escape this @>" , buf_no);
  }
  if ( ! hidden ) {
    macro_name = grab_d_name(line);
    ref_nums[ macro_name ].push_back(num); 
  }
  break;
}

Also defined in: ... 35, 38 ...

Scrap referenced in: 25

If the scrap is not a hidden scrap we will extract the name of the macro called and store the number of the current scrap in the container ref_nums.

« global declarations » 37

 map_t            ref_nums;
string           macro_name;

Also defined in: ... 30, 39 ...

Scrap referenced in: 2


6.2.5 Ending the Scrap Body - @}, @+

 If the following line does not belong to the scrap and the current line is no entry to the list of user defined entries we will store its index in the appropriate container.

« switch the lookahead » 38

case '}': {
  switch (state) {
    case body : {
      (*map_ptr)[ scrap_name ].push_back(buf_no);
      break;
    }
    case uindex : {
      index[grab_d_name(line)];
      break;
    }
    default : {
      error("Unexpected  @}", buf_no - 1 );
    }
  }

  state = doc;
  hidden = 0;
  break;
}

Also defined in: ... 36, 40 ...

Scrap referenced in: 25

If the current line represents an entry to the user-defined list, we will put its name in the container index.

« global declarations » 39

map_t            index;

Also defined in: ... 37, 44 ...

Scrap referenced in: 2

Whenever the lookahead is a '+' we have to decide whether the current line has to be pushed to the user-defined index or not. Therefore, we set a new state: uindex.

« switch the lookahead » 40

case '+' : {
  if ( state == uindex ) {
    index[grab_d_name(line)];
    break;
  }
  if ( state == body && ! hidden ) {
    (*map_ptr)[ scrap_name ].push_back(buf_no);
    state = uindex;
    break;
  }
  error("Unexpected  @+", buf_no );
}

Also defined in: ... 38, 41 ...

Scrap referenced in: 25


6.2.6 Setting the Documentation Language Depending Strings

 If one wants to use the default settings the desired documentation language must be specified after @l. If this is found in the lookahead, we switch to state docdef.

« switch the lookahead » 41

case 'l' : {
  if ( state == doc )
    state = docdef;
  else
    error("Unexpected @l", buf_no );
  break;
}

Also defined in: ... 40, 43 ...

Scrap referenced in: 25

Having read the next line the following code will be executed.
It extracts the name of the documentation language from the whole at_line and sets the variable docLang to the appropriate value.

« set documentation language » 42

if ( state == docdef ) {
  state = doc;
  size_t a = line.find_first_not_of(" \t\n", 1);
  size_t b = line.find_first_of(" \t\n", a);
  dL = line.substr(a, b-a);
  line = line.substr(b);
  if ( dL == "HTML" || dL == "html" )
    docLang = HTML;
  else {
    if ( dL == "LaTeX" || dL == "LATEX" || dL == "latex" )
      docLang = LATEX;
    else
      warning(dL + ": Documentation language not valid \n No special formatting is done", 0) ;
  }
}

Scrap referenced in: 20

If a user wants to specify his own formatting strings he shows this by giving the tag @sc where c is a character as mentioned in the user's guide. These strings may only be given when the state is set to doc.
If so, we just set the state to fdef, which means format definition.

« switch the lookahead » 43

case 's' : {
  if ( state == doc ) {
    state = fdef;
    continue;
  }
  else 
    error("Unexpected @s", buf_no );
  break;
}

Also defined in: ... 41, 46 ...

Scrap referenced in: 25

We read the next line which must contain either the name of the desired documentation language or a string that contains formatting information. The strings controlling the formatting are stored in appropriate variables.

« global declarations » 44

string  begSStrU;
string  endSStrU;
string  headStrU;
string  defStrU;
string  refStrU;
string  numStrU;
string  numSepU;
string  begBStrU; 
string  endBStrU; 
string  macroNumStrU;
string  begMStrU;
string  endMStrU;
string  begLStrU;
string  endLStrU;
string  listStrU;

Also defined in: ... 39, 53 ...

Scrap referenced in: 2

Depending on the state we jump to the following code:

« set formatting strings » 45

if ( state == fdef ) {
  if (line[0] == '}') {
     state = doc;
  }
  else {
    switch (line[0]) {
      case 's': {
        break;
      }
      case '{': {
        break;
      }
      case '}': {
        state = doc;
        break;
      }
      case 'S': {
        switch (line[1]) {
          case '{': {
            begSStrU = line.substr(2, (line.size() - 2));
            break;
          }
          case '}': {
            endSStrU = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@S" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
      case 'H': {
        switch (line[1]) {
          case '|': {
            headStrU = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@H" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
      case 'B': {
        switch (line[1]) {
          case '{': {
            begBStrU = line.substr(2, (line.size() - 2));
            break;
          }
          case '}': {
            endBStrU = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@B" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
  
      case 'M': {
        switch (line[1]) {
          case '{': {
            begMStrU = line.substr(2, (line.size() - 2));
            break;
          }
          case '}': {
            endMStrU = line.substr(2, (line.size() - 2));
            break;
          }
          case '|': {
            macroNumStrU = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@M" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
  
      case 'R': {
        switch (line[1]) {
          case '|': {
            refStrU = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@R" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
  
      case 'D': {
        switch (line[1]) {
          case '|': {
            defStrU = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@D" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
  
      case 'N': {
        switch (line[1]) {
          case '|': {
            numStrU = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@N" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
  
      case 'C': {
        switch (line[1]) {
          case '|': {
            numSepU = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@N" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
  
      case 'L': {
        switch (line[1]) {
          case '{': {
            begLStrU = line.substr(2, (line.size() - 2));
            break;
          }
          case '}': {
            endLStrU = line.substr(2, (line.size() - 2));
            break;
          }
          case '|': {
            listStrU = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@L" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
  
      case 'l': {
        switch (line[1]) {
          case 't': {
            lt = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@l" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
  
      case 'g': {
        switch (line[1]) {
          case 't': {
            gt = line.substr(2, (line.size() - 2));
            break;
          }
          default: {
            warning("@g" + line.substr(2) + "Unknown kind of formatting string", buf_no);
            break;
          }
        }
        break;
      }
  
      default: {
        warning("@s" + line.substr(1) + "Unknown kind of formatting string", buf_no);
        break;
      }
    }
    default_setting = 0;
    continue;
  }
}

Scrap referenced in: 20


6.2.7 Switching Between docmode and scrapmode

 Switching within a scrap to docmode and back is treated as one case. Switching outside a scrap to scrapmode and back is treated as another case. We merely test whether these switches occur in the right state.

« switch the lookahead » 46

case '|' : {
  if ( state != doc )
    error("Unexpected  @|", buf_no );
  break;
}

Also defined in: ... 43, 47 ...

Scrap referenced in: 25

« switch the lookahead » 47

case '$' : {
  if ( state != body )
    error("Unexpected  @$", buf_no );
  else 
    (*map_ptr)[ scrap_name ].push_back(buf_no);
  break;
}

Also defined in: ... 46, 48 ...

Scrap referenced in: 25



6.2.8 Including Files

 Dealing with include files we have to pay attention to the following points:
As we put the whole parser in a special function, we simply can call it recursively whenever a new file has to be included. Doing so we must check whether two files try to include each other thus causing an infinite loop.
If the lookahead indicates that the name of a file to include will follow, we set the local boolean variable include_file.

« switch the lookahead » 48

case 'i' : {
  if ( state == doc ) {
    include_file = 1;
    break;
  }
  error("Unexpected  @i", buf_no );
  break;
}

Also defined in: ... 47, 50 ...

Scrap referenced in: 25

The other tasks mentioned above must be done after the next line is read.
Since the next name must be the name of the file to include, we extract it from the rest of the at_line

« include a file » 49

if ( include_file ) {
  size_t s = line.find_first_not_of(" \t\n", 1);
  size_t e = line.find_first_of(" \t\n", s);
  incl_filename = line.substr(s, e-s);

  if ( mark.find(incl_filename) != mark.end() ) 
    error(incl_filename + ":  recursive call of this file" , buf_no);
  else
    mark.insert(incl_filename);
  string rline = line;
  read_source_file(incl_filename.c_str()); 
  line = rline;
  mark.erase(incl_filename);
  include_file = 0;
}

Scrap referenced in: 20



6.2.9 Other Tags

 As we intended to switch over all possible characters following the at-sign and building a valid tag, we have to list them here. We can not use the default switch because there the fake character is added.

« switch the lookahead » 50

case 'f' : 
case 'm' : 
case 'u' : 
  break;

Also defined in: ... 48

Scrap referenced in: 25





6.3 Extracting the Output Files

 After the first pass all information we need to extract the output files and to generate the documentation is stored in the container and some variables. So we need no further access to any file to proceed until we want to write the generated files.
The generation of the output files is done by the function make_files() as already mentioned. To do so we loop over the container files after we have merged the content of scrap and hide.

« make the files » 51

void make_files() {
  if (wwdebug) wwdb("make_files");
  //merge 'scrap' and 'hide' to copy ALL scraps into files
  std::string sname;
  for (map_t::iterator h_it = hide.begin(); h_it != hide.end(); ++h_it)  {
    sname = h_it->first;
    if ( scrap.find(h_it->first) == scrap.end() ) {
     scrap[h_it->first] = h_it->second;
    }
    else {
     scrap[h_it->first].merge(h_it->second);
    }
  }

  for (map_t::iterator f_it = files.begin(); f_it != files.end(); ++f_it)  {
    <evaluate output file options: 61>
    <prepare the #line directives: 62> 

    mark.clear();
//    const int bS = 100000;
//    char stringBuffer[bS];
//    ostringstream os(stringBuffer, bS);                      
    ostringstream os;
    os << f_it->second;  
    write_or_not(os, f_it->first);
  }
}

Scrap referenced in: 1

and write the content of each file to an ostringstream. Therefore, we overload the operator <<. The second operand is of type text_t that is a list of integers. These integers must be interpreted as indexes to buf where all the lines of the original source are stored.
So, first we have to loop over the list txt and take for each integer the corresponding line from buf.
Then, the advantage of the way we made the partitions of the source becomes obvious: The first character of each line in buf holds the information about the sort of line.

« functions and routines » 52

ostream& operator << (ostream& os, const text_t& txt) {
  if (wwdebug) wwdb("operator <<");
  string scrap_name;
  string blanks;
  int ic;
  for (text_t::const_iterator it = txt.begin(); it != txt.end(); ++it) {
    switch ( buf[*it] [0] ) {
      <switch in accordance with the first character: 54, 57>

    };
  }
  return os;
}

Also defined in: ... 33, 64 ...

Scrap referenced in: 1

If a macro call occurs in the scrap, the line starts with '<'. We extract the macro name and check whether this macro call is recursive and causing an infinite loop. For this purpose we maintain a set of scrap names.

« global declarations » 53

 set< string >    mark;

Also defined in: ... 44, 55 ...

Scrap referenced in: 2

A scrap name is pushed in the set whenever it is called and it will be removed by closing the scrap. So, the first thing we do when we find a macro call is to test whether its name is already included in the set. If so, we give an error message and exit. If not we call the operator again with the index list of the macro.

« switch in accordance with the first character » 54

   case '<' : { 
     scrap_name = grab_d_name(buf[*it]);
     if ( mkind ) {
       <compute offset for indentation: 56>
     }
     if ( mark.find(scrap_name) != mark.end() ) 
       error(scrap_name + ":  recursive macro call" , buf_no);
     else
       mark.insert(scrap_name);
     if ( scrap.find(scrap_name) != scrap.end() ) os << scrap[scrap_name];
      else warning(scrap_name + ": Scrap never defined", buf_no);
     if ( mkind )
       ind_sum = ind_sum - ic;        
     mark.erase(scrap_name);
     break;
   }

Also defined in: , 57 ...

Scrap referenced in: 52

Here the offsets for the indentation of the expanded macros have to be computed. The line preceding the macro call gives the information we need. We only have to find the last newline in this string. The number of characters after this newline constitutes our offset. Therefore, we use the string member function string::rfind() taking as parameter the string to search for. This is a reverse search as it starts at the end of the string in which the search string is to be found. The result, if any, is an index pointing to the beginning of the search string.
Now we only have to add this offset to a former one if it exists. We store the total offset in the global variable ic

« global declarations » 55

int ind_sum = 0;
int offset;

Also defined in: ... 53, 63 ...

Scrap referenced in: 2

and add the computed offset when we expand the macro and substract this offset when we are at the end of this macro. This can comfortably be done because offset is local to the overloaded operator <<.

« compute offset for indentation » 56

       size_t ipos = buf[(*it) -1].rfind('\n');
       size_t b_size = buf[(*it) -1].size();
       if ( ipos == string::npos ) 
         ic = b_size + offset;
       else
         ic  = b_size - ipos - 1;
       ind_sum = ind_sum + ic;
       offset = ic;

Scrap referenced in: 54

All lines beginning with another character must be printed to the ostream by removing the first character.

« switch in accordance with the first character » 57

   case '$' :
   case '{' : 
   case '>' : {
     <print #line: 59>
     for (int i = 1; i < buf[*it].size(); ++i) {
       os << buf[*it][i];
       <print indentation: 58>
     }
     break;
   }
   default :
     break;

Also defined in: ... 54

Scrap referenced in: 52

Whenever we print a newline and mkind is true, the current offset must be printed.

« print indentation » 58

if ( buf[*it][i] == '\n' && mkind ) {
for ( int ici = 0; ici < ind_sum; ++ici ) 
  os << " ";
}

Scrap referenced in: 57

If the boolean variable mkline is true, the #line directive must be printed.

« print #line » 59

nextnumber = 1;
if ( mkline &&  nextnumber ) {
os << "\n#line " << lines[*it] << " \"" << source << "\""  << endl;
nextnumber = everyline;
}

Scrap referenced in: 57

Both boolean variables mentioned above

« hidden declarations » 60

bool mkline = 0;
bool mkind = 1;
bool nextnumber;
bool everyline = 0;

Also defined in: ... 34

Scrap referenced in: 2

are set by evaluating the options given with the output filename. As the description of the first pass has shown (6.2.3), these options are stored for each filename in the container options. For this reason, we only search for the significant characters using again the string::find() function. All other characters in the options string are ignored.

« evaluate output file options » 61

opt = options[f_it->first];
if ( opt.find("l") != string::npos )
   mkline = 1;
else
   mkline = 0;
if ( opt.find("i") != string::npos )
   mkind = 0;
else
   mkind = 1;

Scrap referenced in: 51

The line numbers will only be computed if the option -l is given with one of the output files and if the output files are to be generated. This is in accordance with our objective to make the default behavior of WebWeb as fast as possible.

« prepare the #line directives » 62

if ( mkline && lines.empty() ) 
  compute_line_numbers();

Scrap referenced in: 51

The computing is done as follows:
For each at-line in the source buffer buf we count the number of newlines and store this number in the container lines which is a vector of integers.

« global declarations » 63

line_t           lines;
int line_no;

Also defined in: ... 55, 71 ...

Scrap referenced in: 2

This results in the following routine:

« functions and routines » 64

void compute_line_numbers() {
  if (wwdebug) wwdb("compute_line_numbers");
  line_no = 1; 
  size_t pos;
  for (buf_t::iterator b_it = buf.begin(); b_it != buf.end(); ++b_it) {
    lines.push_back(line_no);
    pos = (*b_it).find('\n');
    while ( pos != string::npos ) {
      ++line_no;
      pos = (*b_it).find('\n', pos + 1);
    }
  }
}

Also defined in: ... 52, 65 ...

Scrap referenced in: 1

In case of an error or a warning, the actual line number is computed by adding the number of newlines in the current line read by getline so far.

« functions and routines » 65

int actual_line_number() {
  if (wwdebug) wwdb("actual_line_number");
  size_t pos = line.find('\n');
  while ( pos != string::npos ) {
    ++line_no;
    pos = line.find('\n', pos + 1);
  }
return(++line_no);
}

Also defined in: ... 64, 66 ...

Scrap referenced in: 1




6.4 Writing the File


 After these procedures of scanning and parsing the ostringstream os contains the whole content of the source file. To write it physically to the file, we use the function write_or_not(). taking the ostringstream as a reference and the name of the outputfile as parameters. After some tests it opens an output file stream ofstream ofs and redirects the given ostringstream to it.

« functions and routines » 66

void write_or_not(ostringstream& ostst, const string& fname) {
  if (wwdebug) wwdb("write_or_not");
  bool writefile = force;
  string line1;
  string line2;
  string answer;
  bool repeatit;
  bool gl1;
  bool gl2;

  <check whether file has changed: 68>

  if ( interactive ) {
    <ask whether user wants to overwrite: 67>
  }

  if ( writefile ) {
    ofstream ofs(fname.c_str());
    if ( ! ofs.good() )
      error("Cannot write to file " + fname, 0);
    ofs << ostst.str(); 
  }
}

Also defined in: ... 65, 69 ...

Scrap referenced in: 1

The tests are carried out to ensure that an existing file is only overwritten in the case it has changed and the user wants to overwrite it. This behavior is controlled with the command line options as listed in the How to use section of this document. So if the user has started WebWeb in the interactive-mode we would set the variable writefile in accordance with the user input. This mode has higher priority than the force-mode displayed with the variable force which normally writes the files without checking whether they have changed or not.

« ask whether user wants to overwrite » 67

if ( writefile ) {
  cout << "File " << fname << " has changed. Overwrite? [y|n] :";
  cin >> answer;
}
else {
  cout << "File " << fname << " has not changed. Overwrite anyway? [y|n] :";
  cin >> answer;
}
if ( answer == "y" || answer == "yes" || answer == "Y" || answer == "YES" )
  writefile = 1;
else
  writefile = 0;

Scrap referenced in: 66

If no command option is given we will have to check whether the generated file already exists. For this purpose, we open the input file stream ifs on the generated file. If this is possible, the file exists and we redirect the new generated file that is still stored in the ostringstream to an istringstream. Otherwise we simply write the content of the ostringstream to the file. In the first case, we read both files parallel linewise as long as they are identical or the end of one file is reached. According to the result we set the variable writefile which is used to make the final decision on writing the file or not in the function write_or_not().

« check whether file has changed » 68

if ( ! writefile ) {
  ifstream ifs(fname.c_str());
  istringstream iostst(ostst.str());     
  repeatit = ifs.good(); 
  if ( !repeatit )
    writefile = 1;
  while ( repeatit ){
    gl1 = getline(ifs, line1);
    gl2 = getline(iostst, line2);
    if ( !gl1 && !gl2 )
      repeatit = 0; 
    else {
      if ( (gl1 && gl2 && ( line1 != line2 )) || (!gl1 || !gl2) ){
        repeatit = 0;
        writefile = 1;
      }
    }
  }      
}

Scrap referenced in: 66



6.5 Generating the Documentation


 In order to generate the documentation, we maintain some functions. The following list gives an overview of these functions:

« functions and routines » 69

<header_line(): returns scrap header: 90>
<formatted_list(): returns formatted list of scrap numbers: 85>
<ref_list(): returns Scrap referenced in: + scrap_numbers: 88>
<def_list(): returns Also defined in: + scrap_numbers: 87>
<macro_num_list(): returns def_list() formatted for using inside a scrap: 89>
<print_formatted_list(): prints any kind of formatted list: 86>
<init_formatting_strings(): part of language independence: 8>
<print_line(): searchs in given line for user index and prints the line: 91>
<make_doc(): The main-function for documentation generation: 72>

Also defined in: ... 66

Scrap referenced in: 1

We start describing these functions first by showing how make_doc() works. Then, we take a look at the other functions.

6.5.1 How the generation of the documentation works

 By scanning the source file we have stored most of the text in the container buf.Based on the fact that the text is stored in the same order as the source, we draw the concept how to generate the documentation:
We simply scan the whole contents of buf.
The behavior and kind of formatting depends on the first character in each at-line. This is why we switch in accordance with the first character.
In addition to special formatting of some parts it is necessary in some cases to print extra information, e.g. the list of the numbers of scraps defining the same macro.

First, however, we have to apply some initial tasks:
  1. In the first pass we collected for each macro the numbers of all scraps calling this macro. But when the same macro was called twice in the same scrap we pushed this scrap number twice in ref_nums. Since we do not want a number listed twice or even more we have to remove them. In this context, we use list::unique, another comfortable STL function.
  2. Again, we need a variable which holds the state and allows us to determine whether we are in a header, a scrap, or outside. We initialize the state to doc.
  3. To determine whether a at_line is visible or not , we set a boolean variable. This is used for hidden scraps.

« do initial tasks » 70

init_formatting_strings();

for (map_t::iterator dit = ref_nums.begin(); dit != ref_nums.end(); ++dit)  
  dit->second.unique();

string s;
char_t c;
state = doc;
bool visible = 1;

Scrap referenced in: 72

The boolean variable visible indicates whether a at-line has to be printed or not. It is true by default but is set to false when a hidden scrap is found and set back to true when the end of the scrap is reached.

At the end, we call again the routine write_or_not which was described in detail in the section Write the file. So we can use the same mechanism for controlling the overwriting of the documentation file and for the output files. We write the lines not directly to the file but in an ostringstream. Thus, we can use it later to test whether the generated documentation differs from an existing one or not.

« global declarations » 71

ostringstream    ds(1000000);

Also defined in: ... 63, 73 ...

Scrap referenced in: 2

By taking these steps we get the following basic structure when we generate the documentation:

« make_doc(): The main-function for documentation generation » 72

void make_doc() {
  if (wwdebug) wwdb("make_doc");
  ds.seekp(0, ios::beg);
  <do initial tasks: 70>

  for (buf_t::iterator b_it = buf.begin(); b_it != buf.end(); ++b_it) {
    c = tolower(static_cast<char_t>((*b_it)[0]));

    if ( visible ) {
      switch ( c ) { 
        <evaluate first character in at_line: 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84>
      }
    }
    else {
      if ( c == '}' ) {
         print_line(*b_it);
         state = doc;
         visible = 1;
      }
   }
 } 
write_or_not(ds, docfile);
}

Scrap referenced in: 69


6.5.2 Doing the at-line-specific tasks

Scrap Headers


Let us first take a look at the at-lines which define a scrap header. In these cases we must print the formatted header line. Therefore, we call the function header_line and give the arguments scrap name, the current scrap number

« global declarations » 73

int s_num = 0;

Also defined in: ... 71

Scrap referenced in: 2

and the HeaderString which contains the formatting information.

« evaluate first character in at_line » 74

case 'o' : {
  if ( state == doc ) {
    ds << *begSString;
    ds << header_line(scrap_name = grab_o_name(*b_it), ++s_num, *headerString);
  }
  break;
}
case 'd' : {
  if ( state == doc ) {
    ds << *begSString;
    ds << header_line(scrap_name = grab_d_name(*b_it), ++s_num, *headerString);
  }
  break;
}

Also defined in: , 75 ...

Scrap referenced in: 72

Since we do not want to copy a hidden scrap to the documentation we set the variable visible to false in regard with the character h.

« evaluate first character in at_line » 75

case 'h' : {
  if ( state == doc ) 
    visible = 0;
  break;
}

Also defined in: ... 74, 76 ...

Scrap referenced in: 72



Scrap Bodies

Because the body of a scrap may be printed in a special format for layout reasons, we print the formatting bodyStartString whenever a scrap starts. After that we print the rest of the at_line.

« evaluate first character in at_line » 76

case '{' : {
  ds << *bodyStartString;
  ds << *toScrapmodeString;
  state = body;
  print_line(*b_it);
  break;
}

Also defined in: ... 75, 77 ...

Scrap referenced in: 72

Of course, we must print the appropriate bodyEndString when we reach the end of a scrap.
Then, we have to print the list of scrapnumbers defining the same macro and the list of numbers of the scraps that call this macro.

« evaluate first character in at_line » 77

 case '}' : {
  if ( state == body ) {
    state = doc;
    ds << *toDocmodeString;
    ds << *bodyEndString;
    ds << def_list(scrap_name, s_num, *defString);
    ds << ref_list(scrap_name, s_num, *refString);
    ds << *endSString << endl;
  }
  print_line(*b_it);
  break;
}

Also defined in: ... 76, 78 ...

Scrap referenced in: 72



Macro Calls

With respect to the start of a macro call, we have to distinguish between a macro call inside a scrap and a macro call in the documentation section. In both cases, we first extract the macro_name using again the function grab_d_name().
If we are in the documentation section, the macro must be stored in the container hide. So we use the overloaded operator << again to print the whole extracted content of the macro.

In order to identify the macro call in the documentation we have to print the <-sign in front of the macro name. Because it depends on the documentation language used we print the string lt which is set to the current representation. After the macro name the list of all scraps defining this macro is printed.

« evaluate first character in at_line » 78

case '<' : {
  macro_name = grab_d_name(*b_it);
  if (state == doc) {
    if ( hide.find(macro_name) != hide.end() ) 
        ds << hide[macro_name];
    else ds << *b_it; 
  }
  if (state == body) {
    if ( scrap.find(macro_name) != scrap.end() )  
      ds << *startMacroCall << macro_name << macro_num_list(macro_name);
  } 
  break;
}

Also defined in: ... 77, 79 ...

Scrap referenced in: 72

At the end of a macro call we must print the >-sign if we are in a scrap and in both cases the rest of the at_line.

« evaluate first character in at_line » 79

case '>' : {
  if ( state == body ) {
    if ( scrap.find(macro_name) != scrap.end() )  
    //if ( hide.find(macro_name) == hide.end() ) 
      ds << *endMacroCall;
  }
  print_line(*b_it);
  break;
}

Also defined in: ... 78, 80 ...

Scrap referenced in: 72



Scrapmode and Docmode

WebWeb provides the possibility of displaying parts of the scraps in the same format as the documentation called docmode and vice versa called scrapmode.
We set the state to docmode or body depending on the fact whether we are at the beginning or at the end of the docmode section. After printing the formatting string we print the rest of the at-line.

« evaluate first character in at_line » 80

case '$' : {
  if ( state == docmode ) {
    ds << *toScrapmodeString;
    state = body;
  }  
  else { 
    if ( state == body ) {
      ds << *toDocmodeString;
      state = docmode;
    }  
  }
  print_line(*b_it);
  break;
}

Also defined in: ... 79, 81 ...

Scrap referenced in: 72

If we are in the documentation and want to switch to the scrapmode we set the state to it and apply the same tasks as described above.

« evaluate first character in at_line » 81

case '|' : {
  switch (state) {
    case doc: {
      ds << *toScrapmodeString;
      state = scrapmode;
      break;
    }
    case scrapmode: {
      ds << *toDocmodeString;
      state = doc;
      break;
    }
    default: {
      error("Switch between docmode and scrapmode is wrong placed", 0);
      break;
    }
  }
  print_line(*b_it);
  break;
}

Also defined in: ... 80, 82 ...

Scrap referenced in: 72



The Index Lists

We have the possibility of printing three index lists. In all cases we proceed in the same way:
We call the function print_formatted_list(). Then we print the rest of the at_line. The only difference consists of the parameters we give when calling the function.

« evaluate first character in at_line » 82

case 'f' : {
  print_formatted_list(files, def_nums);
  print_line(*b_it);
  break;
}
case 'm' : {
  print_formatted_list(scrap, def_nums);
  print_line(*b_it);
  break;
}
case 'u' : {
  print_formatted_list(index, index);
  print_line(*b_it);
  break;
}
case '+' : {
  break;
}

Also defined in: ... 81, 83 ...

Scrap referenced in: 72



Remove the Name of Included Files

As demonstrated the parser copies the at_line in which the file is called to buf when it has finished reading an include file. Thus, this at-line starts with the character i and we must remove the filename.
Because print_line() removes the first character from any given string, we remove one character less when we remove the filename. So we do not need to alter the convention of calling print_line() in each case.

« evaluate first character in at_line » 83

case 'i' : {
  size_t a = (*b_it).find_first_not_of(" \t\n", 1);
  size_t b = (*b_it).find_first_of(" \t\n", a);
  print_line((*b_it).substr(b));
  break;
}

Also defined in: ... 82, 84 ...

Scrap referenced in: 72



First Lines And the Default

Since the parser put a 'z' in front of any first line of any source file, we have to remove it. Although it is not necessary to list this case because it would be dealt with by the default, we list it here in order to contribute to a better understanding of the whole procedure.

« evaluate first character in at_line » 84

case 'z' : {
  print_line(*b_it);
  break;
}
    
default : {
  print_line(*b_it);
  break;
}

Also defined in: ... 83

Scrap referenced in: 72




6.6 Other Functions Needed to Generate the Documentation

 The following section describes the functions listed at the beginning of the chapter Generating the Documentation.
They are not only responsible for the language specific work but also aim at making the program readable and to easily carrying out modifications. Most of them are called from different states of make_doc() with different arguments, some of them call others, and some of them are never called directly from make_doc().


6.6.1 formatted_list()

 One of the basic features of literate programming are the crossreferences providing the possibility of reading the document in an individual order. Therefore we attach to each scrap the references to these scraps which define the same macro and the references to the scraps which reference this macro.
In order to make it possible to read and use these crossreferences we often have to print a formatted list of references. As we number the scraps this list contains integers and we use a function to generate this formatted lists. It needs two arguments: For each integer in the list which is not equal to s_num we replace the metaNumber in the string numString which holds the formatting information and append this string to listOfNums which is the return value. In addition, we provide a numSeparator (usually a comma) which is copied between every two numbers.

« formatted_list(): returns formatted list of scrap numbers » 85

string formatted_list(const text_t& locTxt, const int& s_num) {
  if (wwdebug) wwdb("formatted_list");
  string listOfNums = "";
  string tempString;
  size_t numPos;
  char_t number [5];
  for (text_t::const_iterator it = locTxt.begin(); it != locTxt.end(); ++it) { 
    if ( *it != s_num ) {
      if ( listOfNums != "" )
        listOfNums += (*numSeparator);
      sprintf(number, "%d", *it); 
      tempString = *numString;
      numPos = tempString.find(metaNumber);
      while ( numPos != string::npos ) {
        tempString.replace(numPos, metaNumber.length(), number);
        numPos = tempString.find(metaNumber);
      }
      listOfNums += tempString;
    }
  }
  return listOfNums;
}
string formatted_list2(const text_t& locTxt, const int& s_num) {
  if (wwdebug) wwdb("formatted_list");
  //  sort(logTxt.begin(), logTxt.end());
  string listOfNums = "";
  string listOfNums1 = "";
  string listOfNums2 = "";
  int counter1 = 0;
  int counter2= 0;
  string tempString;
  size_t numPos;
  char_t number [5];
  bool flag1 = true;
  bool flag2 = true;
  for (text_t::const_iterator it = locTxt.begin(); it != locTxt.end(); ++it) { 
    if ( *it == s_num ) {
       flag1 = false;
       continue;
    }
    if ( flag1 ) {
      listOfNums1 = "...";
      sprintf(number, "%d", *it); 
      tempString = *numString;
      numPos = tempString.find(metaNumber);
      while ( numPos != string::npos ) {
        tempString.replace(numPos, metaNumber.length(), number);
        numPos = tempString.find(metaNumber);
      }
      listOfNums1 += tempString;
    }
    if ( (! flag1) && (flag2) ) {
      listOfNums2 = (*numSeparator);
      sprintf(number, "%d", *it); 
      tempString = *numString;
      numPos = tempString.find(metaNumber);
      while ( numPos != string::npos ) {
        tempString.replace(numPos, metaNumber.length(), number);
        numPos = tempString.find(metaNumber);
      }
      listOfNums2 += tempString;
      listOfNums2 += " ...";
      flag2 = false;
    }
  }
  listOfNums = listOfNums1 + listOfNums2;
  return listOfNums;
}

Scrap referenced in: 69


6.6.2 print_formatted_list()


 This function is used to make the list of files, the list of macros and the user-defined index. It needs two arguments: First, it starts the list environment printing the listStartFormat. Then, it prints for each entry in the first argument the list of references. For this purpose, it takes the listEntryFormat and replaces the metaName with the current key from locMap using the memberfunction string::replace(). Then, it replaces in the same string the metaList with the returned string from a call to formatted_list() which is called with the third argument set to empty because we want all entries from the index list.
After all these manipulated strings are copied to the ostringstream ds, the list environment is closed by the listEndFormat.

« print_formatted_list(): prints any kind of formatted list » 86

void print_formatted_list(const map_t& locMap, map_t& ef_nums) {
  if (wwdebug) wwdb("print_formatted_list");
  ds << *listStartFormat;
  string locEntryF;
  size_t namePos = listEntryFormat->find(metaName);
  size_t listPos;
  if  ( namePos != string::npos ) {
    for ( map_t::const_iterator it = locMap.begin(); it != locMap.end(); ++it) {
      locEntryF = *listEntryFormat;
      locEntryF.replace(namePos, metaName.length(), it->first);
      listPos  = locEntryF.find(metaList);
      if ( listPos != string::npos ) 
        ds << locEntryF.replace(listPos, metaList.length(), formatted_list(ef_nums[it->first], 0)); 
    }
  }
  ds << *listEndFormat; 
}

Scrap referenced in: 69


6.6.3 def_list() and ref_lists()

 These functions print two lists under each scrap: Both are called from make_doc() when a scrap has finished and need three arguments: If the defStringLocal is empty or there are not enough numbers in the list, the empty string will be returned. Otherwise the metaList is replaced by the formatted_list() of the appropriate integers.

« def_list(): returns Also defined in: + scrap_numbers » 87

string def_list(const string& s, const int& s_num, string defStringLocal) {
  if (wwdebug) wwdb("def_list");
  size_t listPos;
  string rString;
  if ( (def_nums[s]).size() >= 2 && defStringLocal.length() != 0) {
    listPos  = defStringLocal.find(metaList);
    if  ( listPos != string::npos ) 
      defStringLocal.replace(listPos, metaList.length(), formatted_list2(def_nums[s], s_num));
    rString = defStringLocal;
  }
  return rString;
}

Scrap referenced in: 69

The function ref_list does an equivalent task for the scraps calling the current macro.

« ref_list(): returns Scrap referenced in: + scrap_numbers » 88

string ref_list(const string& s, const int& s_num, string refStringLocal) {
  if (wwdebug) wwdb("ref_list");
  size_t listPos;
  string rString;
  if ( ! (ref_nums[s]).empty() && refStringLocal.length() != 0) {
    listPos  = refStringLocal.find(metaList);
    if  ( listPos != string::npos ) 
      refStringLocal.replace(listPos, metaList.length(), formatted_list(ref_nums[s], s_num));
    rString = refStringLocal;
  }
  return rString;
}

Scrap referenced in: 69


6.6.4 macro_num_list()

 macro_num_list() returns the list appended to each macroname which is called inside a scrap.
It is similar to def_list(), structured in a simplier way, though, since we want to print all numbers in the list and need only the current scrapname as argument

« macro_num_list(): returns def_list() formatted for using inside a scrap » 89

string macro_num_list(const string& s) {
  if (wwdebug) wwdb("macro_num_list");
  string tempString = *macroCallNum;
  size_t listPos = tempString.find(metaList);
  if ( listPos != string::npos ) 
    tempString.replace(listPos, metaList.length(), formatted_list(def_nums[s], 0));
  return tempString; 
}

Scrap referenced in: 69


6.6.5 header_line()

 At the beginning of each scrap we print the special formatted scrapname and append the current scrapnumber. To generate this formatted header we use header_line() based on three parameters: First, we replace all occurences of the metaName with the scrapname. Then, we replace all occurences of the metaNumber with the current scrapnumber by strongly referring to the STL. Finally, we return the so manipulated string.

« header_line(): returns scrap header » 90

string header_line(const string& s, const int& s_num, string headerStringLocal) {
  if (wwdebug) wwdb("header_line");
  if ( ! headerStringLocal.empty() ) {
    size_t namePos = headerStringLocal.find(metaName);
    while ( namePos != string::npos ) 
      namePos = ( headerStringLocal.replace(namePos, metaName.length(), s) ).find(metaName); 
    char_t number [5];
    sprintf(number, "%d", s_num); 
    size_t numPos = headerStringLocal.find(metaNumber);
    while ( numPos != string::npos ) 
      numPos = (headerStringLocal.replace(numPos, metaNumber.length(), number)).find(metaNumber);
  }
  return headerStringLocal;
}

Scrap referenced in: 69

6.6.6 print_line()

 The function print_line() is mainly used to remove the first character in an at_line.

« print_line(): searchs in given line for user index and prints the line » 91

void print_line(string s) {
  if (wwdebug) wwdb("print_line" + s);
  string::iterator  pli = s.begin() +1;
  size_t spos;
  string tmp_string;
  string tmp_schar;
 
  s = s.substr(1, (s.length() - 1));
  switch ( docLang ) {
    <CASE HTML: 92>
    <CASE LATEX: 93>
    default : {
        ds << s;
    }        
  }
  <search for index: 94>
}

Scrap referenced in: 69

Additionally, it provides some default actions for language specific tasks as replacing some special characters inside a scrap. For example, the < - sign which is to be replaced in a HTML-document by &lt;

« CASE HTML » 92

case HTML : {
  if ( (state == body) || (state == scrapmode) ) {
    spos = s.find('<');
    while  ( spos != string::npos ) {
      s.replace(spos, 1, lt);
      spos = s.find('<', spos);
    }
    spos = s.find('>');
    while ( spos != string::npos ) {
      s.replace(spos, 1, gt);
      spos = s.find('<', spos);
    }
  }
  ds << s;
  break;
}

Scrap referenced in: 91

In LaTeX the switching to documentation mode and back inside a scrap is a little bit more difficult. We can not use the simple verbatimenvironment here, because switching to this environment and back implies a vskip and some more directives. For that reason we put a simple \verb@ and @\\ around each line.

« CASE LATEX » 93

case LATEX : {
  if ( (state == body) || (state == scrapmode) ) {
    string latex_at = "@\\verb|@|\\verb@";
    string latex_nl = "@\\newline \n \\verb@";
    spos = s.find('@');
    while  ( spos != string::npos ) {
      s.replace(spos, 1, latex_at);
      spos = s.find('@', (spos + latex_at.length()));
    }
    spos = s.find('\n');
    while  ( spos != string::npos ) {
      s.replace(spos, 1, latex_nl);
      spos = s.find('\n', (spos + latex_nl.length()));
    }
  } 
  ds << s;
  break;
  }

Scrap referenced in: 91

If an user-defined index entry exists, we have to search for it in all lines inside a scrap to find the references to all entries. This is done by print_line(). For each entry in index we use string::find() to test whether the index entry is in the current at_line. If so, we push_back the current scrapnumber to the list <int> which is the value of the current index entry.

« search for index » 94

size_t tmp_pos;
if ( state == body && ! index.empty() ) {
  for ( map_t::iterator plm_it = index.begin(); plm_it != index.end(); ++plm_it) {
    tmp_pos = s.find(plm_it->first);
    if ( tmp_pos != string::npos )  
      plm_it->second.push_back(s_num);
  }
}

Scrap referenced in: 91





7 Compiling

 We compiled WebWeb using the Visual C++ compiler from Microsoft under Windows NT.
We put the command line in a batch file in order to prevent from typing this line during the development of WebWeb again and again.

« mkweb.bat » 95

cl -GX -GR webweb.cpp




7.1 The Preprocessor Directives

 As every C++ program uses library routines we have to include them.

« preprocessor directives » 96

#include < iostream >
#include < fstream >
#include < sstream >
#include < string >

#include < vector >
#include < list >
#include < set >
#include < map >
#include < algorithm >

#ifndef __GNUC__
  using namespace std;
#endif

Scrap referenced in: 2





8 The User's Guide

 It always seemed useful to me having a compact reference manual when working with a new tool. So I decided to make such a special user's guide using some tables to struct the main features.
You can find it in the file wwguide.html in the directory where the file webweb.ww resides.

« wwguide.html » 97

<HTML>
<HEAD>
<Title>WebWeb</Title>
</HEAD>
<BODY>



</BODY>





Appendix A: List of files


mkweb.bat : 95
webstyles.css : 98
webweb.cpp : 1
webweb.h : 2
wwguide.html : 97





Appendix B: List of Macros
CASE HTML : 92
CASE LATEX : 93
analyze the source line : 25
ask whether user wants to overwrite : 67
check whether file has changed : 68
compute offset for indentation : 56
def_list(): returns Also defined in: + scrap_numbers : 87
do initial tasks : 70
evaluate arguments : 10, 13, 14, 17
evaluate first character in at_line : 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84
evaluate output file options : 61
formatted_list(): returns formatted list of scrap numbers : 85
functions and routines : 11, 32, 33, 52, 64, 65, 66, 69
global declarations : 4, 5, 6, 7, 12, 15, 16, 19, 21, 26, 29, 30, 37, 39, 44, 53, 55, 63, 71, 73
header_line(): returns scrap header : 90
hidden declarations : 27, 34, 60
include a file : 49
init_formatting_strings(): part of language independence : 8
macro_num_list(): returns def_list() formatted for using inside a scrap : 89
make the files : 51
make_doc(): The main-function for documentation generation : 72
prepare the #line directives : 62
preprocessor directives : 96
print #line : 59
print indentation : 58
print_formatted_list(): prints any kind of formatted list : 86
print_line(): searchs in given line for user index and prints the line : 91
read the input files : 18
read the source : 20
ref_list(): returns Scrap referenced in: + scrap_numbers : 88
search for index : 94
set documentation language : 42
set formatting strings : 45
skip comments : 24
some typedefs : 3
substitute double at-sign : 23
switch in accordance with the first character : 54, 57
switch the lookahead : 28, 31, 35, 36, 38, 40, 41, 43, 46, 47, 48, 50
the main function : 9
work on first line : 22




Appendix C: Differences between nuweb and WebWeb

These are the major differences between the literate programming tools nuweb and WebWeb:
  • @c can be used for comments. The comment must be closed with @}. So it is simple to comment out a whole scrap by changing @d, @o, or @h to @c.
  • A scrap introduced by @h (hide) and the macro calls referring to it will not be written to the documentation. The macro calls, however, will be expanded in the output files.
  • It is possible to use macros in the documentation sections, too. They are called as usual and defined by @h.
  • Each index listed at the end of a scrap must be introduced by @+. So it is possible to write all sorts of strings to the user-defined index. This list must be followed by the @} closing the scrap.
  • Input files must have the extension .ww.
  • The default documentation language is HTML. Therefore the generated documentation has the extension .html.
  • Documentation text embraced by @| will be displayed in the same font as the scraps.
  • Scrap text embraced by @$ will be displayed in the same font as the documentation text.
  • It is possible to give the name of the documentation file to be generated. This is done with the option -N filename.
  • WebWeb checks each file that is to be written to disk whether it already exists. If so, it will be only written if it has changed.
    This behavior preserves the timestamp on files that have not changed and may be used for projects using makefiles.
    The option -f forces the writing of the files even if they have not changed. Using this option speeds up the running of WebWeb as file changes will not be checked.
    The option -i provides an interactive mode. Each output file is checked for changes. The user gets the result and will then be asked whether to overwrite or not.
  • WebWeb provides a mechanism to change the layout of the documentation by simply changing one of the formatting strings.






Appendix D: Example

 After all these rules for using WebWeb let us look at an example.
Consider the following short web file:

<HTML>
<HEAD>
<tITLE>WebWeb</TITLE>
</HEAD>
<BODY>

@l HTML

This is a short file to test the main features 
of WebWeb.<BR>
First, let us define an output file:

@o sesame.cpp
@{
These are some declarations
@<a special functions@>
@<the main function@>
@}

Now, we want to describe the functionality of the main function
followed by a listing

@d the main function
@{void main(){
  //some code
}
@}

Then, we make a detailed explanation concerning the superiority of sesame
compared to other programs and list the special function:

@d a special function
@{void special(ernie e, bert b){
  // some code
@<more code@>
}
@}

Thereafter, we list more code
@d more code
@{  // some more code
@}

Finally, we comment the rest of the code,
give a listing,
@d more code
@{  // the rest
@}

and say bye.
</BODY>
Having run WebWeb on this file you get the following documentation:
This is a short file to test the main features of WebWeb.
First, let us define an output file:


« sesame.cpp » 1
These are some declarations
<a special function: 3>
<the main function: 2>
Now, we want to describe the functionality of the main function followed by a listing


« the main function » 2
void main(){
  //some code
}
Scrap referenced in: 1

Then, we make a detailed explanation concerning the superiority of sesame compared to other programs and list the special function:


« a special function » 3
void special(ernie e, bert b){
  // some code
<more code: 4 5>
}
Scrap referenced in: 1

Thereafter, we list more code


« more code » 4
  // some more code
Also defined in: 5
Scrap referenced in: 3

Finally, we comment the rest of the code, give a listing,


« more code » 5
  // the rest
Also defined in: 4
Scrap referenced in: 3

and say bye.
At last a listing of the file sesame.cpp as it was generated by WebWeb: 


These are some declarations

void special(ernie e, bert b){
  // some code
  // some more code
  // the rest
}

void main(){
  //some code
}




Appendix E: CSS - The STYLE-file

 We provide the following CLASSes to ensure easy changing of the basic layout of the scraps. They are in the file webstyles.css which is copied by WebWeb to the current working directory.

« webstyles.css » 98

.clsScrpStrt    {
                }
.clsScrpEnd     {
                  margin-top: 1em;
                }
.clsScrpHd      { 
                  color: #660000;
                  font-weight: 700;
                }
.clsScrpHdA     {
                }
.clsScrpMd     { 
                  margin-left: 3em;
                  margin-bottom: 1em;
                  color: #880000;  
                  display: inline;
                }
.clsMcrCll      { 
                  font-weight: 700;
                }
.clsScrpDf      { 
                  margin-top: -1em;
                  font-style: italic;  
                  font-size: small;
                }
.clsScrpNumHrf  { 
                  text-decoration: none;
                }
.clsScrpRf      { 
                  margin-top: -1em;
                  font-style: italic; 
                  font-size: small;
                }
.clsTbl         { color: #000000;
                  margin-bottom: 2em;
                }
.clsTblHrf      { text-decoration: none;
                  font-weight: 500;
                }
.clsTblNm       { font-weight: 700;
                  color: #660000;
                }
.clsTblEntry    { font-weight: 500;
                  
                }





Appendix F: References

[1] Ulrich Breymann. Die C++ Standard Template Library.
Addison-Wesley, 1996

[2] Preston Briggs. nuweb Version 0.87b. A Simple Literate Programming Tool.
Submitted to IEEE Software, August 1992. Available through anonymous ftp:
Available through:
 ftp://ftp.uni-stuttgart.de/pub/tex/web/nuweb/

[3] Nikos Drakos. All about LATEX2HTML, November 1996,
http://cbl.leeds.ac.uk/nikos/tex2html/doc/latex2html/latex2html.html

[4] Nicolai Josuttis. Die C++ Standardbibliothek.
Addison-Wesley, 1996.
http://www.bredex.de/literatur.html

[5] Ira Pohl. C++ Distilled: a concise ANSI/ISO reference and style guide.
Addison-Wesley, 1997.

[6] Silvio Levy and Donald E.Knuth. CWEB user manual: The CWEB system of structured documentation.
Technical Report STAN-CS-83-977, Stanford University, Oktober 1990.

[7] Donald E. Knuth. Literate programming.
The computer Journal, 27(2):97-111, May 1984

[8] Donald E. Knuth.
Homepage

[9] John D. Ramsdell. sweb v 2.0 Simple support for literate programming in Lisp, 1994.
Available through:
ftp://ftp.uni-stuttgart.de/pub/tex/web/schemeweb/

[10] Norman Ramsey. Literate-programming tools need not be complex.
Submitted to IEEE Software, August 1992. Available through:
 ftp://ftp.uni-stuttgart.de/pub/tex/web/noweb/

[11] Lincoln D. Stein. How to Set Up and Maintain a WebSite.
Addison-Wesley, 1997