
- •Introduction
- •Introduction
- •License
- •Structure of the document
- •Installation
- •Getting TPG
- •Requirements
- •TPG for Linux and other Unix like
- •TPG for M$ Windows
- •TPG for other operating systems
- •Tutorial
- •Introduction
- •Reading the input and returning values
- •Embeding the parser in a script
- •Conclusion
- •Usage
- •Package content
- •Command line usage
- •Grammar structure
- •TPG grammar structure
- •Comments
- •Options
- •Lexer option
- •Word bondary option
- •Regular expression options
- •Python code
- •Syntax
- •Indentation
- •TPG parsers
- •Methods
- •Rules
- •Lexer
- •Regular expression syntax
- •Inline tokens
- •Token matching
- •Splitting the input string
- •Matching tokens in grammar rules
- •Parser
- •Declaration
- •Grammar rules
- •Parsing terminal symbols
- •Parsing non terminal symbols
- •Starting the parser
- •In a rule
- •Sequences
- •Alternatives
- •Repetitions
- •Precedence and grouping
- •Actions
- •Abstract syntax trees
- •Text extraction
- •Object
- •Context sensitive lexer
- •Introduction
- •Grammar structure
- •CSL lexers
- •Regular expression syntax
- •Token matching
- •CSL parsers
- •Debugging
- •Introduction
- •Verbose parsers
- •Complete interactive calculator
- •Introduction
- •New functions
- •Trigonometric and other functions
- •Memories
- •Source code
- •Introduction
- •Abstract syntax trees
- •Grammar
- •Source code

22 |
CHAPTER 5. GRAMMAR STRUCTURE |
5.4.1Syntax
Before TPG 3, Python code is enclosed in double curly brackets. That means that Python code must not contain to consecutive close brackets. You can avoid this by writting } } (with a space) instead of }} (without space). This syntaxe is still available but the new syntax may be more readable. The new syntax uses $ to delimit code sections. When several $ sections are consecutive they are seen as a single section.
5.4.2Indentation
Python code can appear in several parts of a grammar. Since indentation has a special meaning in Python it is important to know how TPG handles spaces and tabulations at the beginning of the lines.
When TPG encounters some Python code it removes in all non blank lines the spaces and tabulations that are common to every lines. TPG considers spaces and tabulations as the same character so it is important to always use the same indentation style. Thus it is advised not to mix spaces and tabulations in indentation. Then this code will be reindented when generated according to its location (in a class, in a method or in global space).
The figure 5.2 shows how TPG handles indentation.
5.5TPG parsers
TPG parsers are tpg.Parser classes. The grammar is the doc string of the class.
5.5.1Methods
As TPG parsers are just Python classes, you can use them as normal classes. If you redefine the init method, don’t forget to call tpg.Parser. init .
5.5.2Rules
Each rule will be translated into a method of the parser.
5.5. TPG PARSERS |
23 |
Figure 5.2: Code indentation examples
Code in grammars (old |
Code in grammars |
Generated code |
Comment |
|
|
||||
syntax) |
(new syntax) |
|
|
|
|
|
|||
|
|
|
|
|
|
||||
|
|
|
|
|
Correct: these lines |
||||
{{ |
|
$ |
if 1==2: |
if 1==2: |
have four spaces in |
||||
|
common. |
These |
|||||||
|
if 1==2: |
||||||||
|
$ |
print "???" |
print "???" |
spaces are removed. |
|||||
|
print "???" |
||||||||
|
$ |
else: |
else: |
|
|
|
|
||
|
else: |
|
|
|
|
||||
|
$ |
print "OK" |
print "OK" |
|
|
|
|
||
|
print "OK" |
|
|
|
|
||||
|
|
|
|
|
|
|
|
||
}} |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
The new syntax has no |
|
WRONG: it’s a bad |
|||||
{{ |
if 1==2: |
trouble in that case. |
if 1==2: |
idea to start a mul- |
|||||
|
|
tiline |
code |
section |
|||||
|
print "???" |
|
|
print "???" |
|||||
|
else: |
|
|
else: |
on the first |
line |
|||
|
|
|
since |
the |
common |
||||
|
print "OK" |
|
|
print "OK" |
|||||
|
|
|
indentation may be |
||||||
}} |
|
|
|
|
|||||
|
|
|
|
di erent from what |
|||||
|
|
|
|
|
|||||
|
|
|
|
|
you expect. No er- |
||||
|
|
|
|
|
ror will be raised by |
||||
|
|
|
|
|
TPG |
but |
Python |
||
|
|
|
|
|
won’t |
compile |
this |
||
|
|
|
|
|
code. |
|
|
|
|
|
|
|
|
|
Correct: |
indenta- |
|||
{{ |
print "OK" }} |
$ |
print "OK" |
print "OK" |
tion does not mat- |
||||
ter in a one line |
|||||||||
|
|
|
|
|
|||||
|
|
|
|
|
Python code. |
|
|||
|
|
or |
|
|
|
|
|
|
|
|
|
$ print "OK" $ |
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|