fickle 1.00 by Jason Tang (tang@jtang.org) This is a scanner generator program, much like flex(1) is to C. If you have no desire to author Tcl programs, particularly those that manipulate text, fickle is not for you. A passing knowledge of flex or some other lex-like program would be useful, as that fickle uses nearly identical syntax and commands as flex. Two good references are the flex man page and the O'Reilly book 'lex & yacc' by Levine, Mason, and Brown. Examples of working fickle code may be found in the 'examples' directory. See the examples' README for further details. fickle is protected by the GNU general public license. See the file COPYING for details. Differences between fickle and flex are noted within fickle.tcl itself. They are replicated below for your perusal. ;# This program reads in a "fickle specification file", such as ;# 'foo.fcl', and generates a valid Tcl file, such as 'foo.tcl'. The ;# translation is nearly identical to that of GNU flex(1) upon C code. ;# fickle is covered by the GNU General Public License. ;# ;# Be aware of the following differences between flex and fickle: ;# ;# - None of the flex command line options work. To specify a ;# case-insensitive scanner, use the special declaration %NOCASE. ;# To override the default rule (unmatched scanner input echoed to ;# yyout), use %SUPPRESS. ;# - None of the %option declarations work. '%option stack' is ;# enabled by default. ;# - fickle does not support the functions/macros: output, yyless, ;# yymore, yyrestart, yy_*_buffer, REJECT, YY_CURRENT_BUFFER, or ;# YY_DECL; nor does it support the declarations %T, %unused, or ;# %used. Use 'YY_FLUSH_BUFFER' and 'BEGIN INITIAL' to simulate ;# yyrestart(). ;# - Neither yytext nor yyleng have global scope. Their scope is ;# limited to the yylex function. ;# - Text following a closing curly brace within an action is ;# ignored. ;# - Textual substitutions of definitions is kind of blind, and will ;# ignore backslashes preceeding opening braces. For example, if ;# there exists a definition "foo", then it would be substituted ;# into the patterns "{foo}" as well as "\{foo}". ;# - Textual substitutions are performed by order of appearance. ;# Thus, if the result of one substitution creates a pattern that ;# looks like a second definition, then a second substitution ;# occurs. To prevent this behavior, place definitions that might ;# result in creating a valid name higher up in the file. ;# - Unlike flex, %{ and %} carry no special meaning within an ;# action specification. They are only allowed within the ;# definition section. ;# - The start state INITIAL is not necessarily zero. See, this ;# should just be an encouragement for you to use constants rather ;# than hard-coded values! ;# - For the most part flex's regexs are compatible with Tcl's, with ;# the exception of <> which is unsupported. Circumflexes (^) ;# do not quite work right because of how regexp treats line ;# breaks. You may also try using Advanced Regular Expressions ;# (AREs); your mileage may vary. ;# ;# Also note the following: ;# ;# - As a safety measure, fickle will not overwrite a file. You ;# may override this behavior by passing the argument "--force" ;# on the command line. ;# - If no input file is listed on the command line fickle will ;# read from standard input and write to standard output. ;# - By default, yyin and yyout point to stdin and stdout, ;# respectively. Override them as necessary. ;# - Like flex, there is no implicit main function; the programmer ;# must explicitly call yylex. ;# - Actions are copied verbatim to the destination. Thus they must ;# follow the rules of normal Tcl code -- opening and closing ;# paranthesis, for example, must be escaped as necessary. ;# - As a convenience to the programmer, the "%GLOBAL" directive ;# adds 'global' declarations to the top of the yylex function. ;# This is to get around Tcl's variable scoping rules. ;# - If you don't want autogenerated headers to be added to the ;# output use the declaration "%NOHEADERS". ;# - To change the 'yy' prefix to something else use the "%PREFIX" ;# directive. The argument will automagically be downcased. ;# However, the pre-defined macro 'BEGIN' does not have a prefix. ;# To get around this limitation, it takes an optional second ;# parameter which will direct it to the correct parameter. For ;# example, suppose that you set "%PREFIX" to "zz". All internal ;# calls to 'BEGIN' will set "zz" as the second parameter. Any of ;# your code which calls BEGIN will need to pass "zz" as well. If ;# you do not specify a prefix, BEGIN will use the default, "yy". ;# - The internal scanning buffer is 1024 bytes; i.e., regexps will ;# not match anything larger than this. Use the "%BUFSIZE" ;# directive to change this value. flex's buffer is 8192 bytes. ;# - fickle will exhaust its internal buffer prior to calling yywrap. ;# That means regular expressions cannot match across file ;# boundaries. ;# - Take a look at the header of generated files to see ;# prototypes for overridable functions. They are nearly identical ;# to that of flex. Here is a quick summary of fickle's directives: %s - regular start state %x - exclusive start state %BUFSIZE - internal buffer size, defaults to 1024 bytes %GLOBAL - set variables to global scope within yylex %NOCASE - do case-insensitive pattern matching %NOHEADERS - do not copy function headers to generated file %PREFIX - change prefix of all internal functions and variables. this does NOT change BEGIN %SUPPRESS - unmatched input causes an error It is my hope that fickle proves useful for your text parsing needs. I am in the process of writing a bison(1) like program to complement fickle. Drop me an email at the above address if you would like contribute or test the code.