More Examples Of Regular Expressions

Regular expressions provide a very powerful method of defining a pattern, but they are a bit awkward to understand and to use properly. So let us examine some more examples in detail.

We start with a simple yet non-trivial example: finding floating-point numbers in a line of text. Do not worry: we will keep the problem simpler than it is in its full generality. We only consider numbers like 1.0 and not 1.00e+01.

How do we design our regular expression for this problem? By examining typical examples of the strings we want to match:

A pattern is beginning to emerge: The total expression is:
[-+]?[0-9]*\.?[0-9]*
At this point we can do three things:
  1. Try the expression with a bunch of examples like the ones above and see if the proper ones match and the others do not.

  2. Try to make it look nicer, before we start off testing it. For instance the class of characters "[0-9]" is so common that it has a shortcut, "\d". So, we could settle for:

    [-+]?\d*\.?\d*
    

    instead. Or we could decide that we want to capture the digits before and after the period for special processing:

    [-+]?([0-9])*\.?([0-9]*)
    
  3. Or, and that may be a good strategy in general!, we can carefully examine the pattern before we start actually using it.

You see, there is a problem with the above pattern: all the parts are optional, that is, each part can match a null string - no sign, no digits before the period, no period, no digits after the period. In other words: Our pattern can match an empty string!

Our questionable numbers, like "+000" will be perfectly acceptable and we (grudgingly) agree. But more surprisingly, the strings "--1" and "A1B2" will be accepted too! Why? Because the pattern can start anywhere in the string, so it would match the substrings "-1" and "1" respectively!

We need to reconsider our pattern - it is too simple, too permissive:

Before trying to write down the complete regular expression, let us see what different forms we have:

Now the synthesis:

(^|[ \t])([-+]?([0-9]+|\.[0-9]+|[0-9]+\.[0-9]*))($|[^+-.])
Or:
(^|[ \t])([-+]?(\d+|\.\d+|\d+\.\d*))($|[^+-.])
The parentheses are needed to distinguish the alternatives introduced by the vertical bar and to capture the substring we want to have. Each set of parentheses also defines a substring and this can be put into a separate variable:
regexp {.....} $line whole char_before number nosign char_after

#
# Or simply only the recognised number (x's as placeholders, the
# last can be left out
#
regexp {.....} $line x x number

Tip: To identify these substrings: just count the opening parentheses from left to right.

If we put it to the test:

set pattern  {(^|[ \t])([-+]?(\d+|\.\d+|\d+\.\d*))($|[^+-.])}
set examples {"1.0"    " .02"  "  +0."
              "1"      "+1"    " -0.0120"
              "+0000"  " - "   "+."
              "0001"   "0..2"  "++1"
              "A1.0B"  "A1"}
foreach e $examples {
    if { [regexp $pattern $e whole \
              char_before number digits_before_period] } {
        puts ">>$e<<: $number ($whole)"
    } else {
        puts ">>$e<<: Does not contain a valid number"
    }
}
the result is:
>>1.0<<: 1.0 (1.0)
>> .02<<: .02 ( .02)
>>  +0.<<: +0. ( +0.)
>>1<<: 1 (1)
>>+1<<: +1 (+1)
>> -0.0120<<: -0.0120 ( -0.0120)
>>+0000<<: +0000 (+0000)
>> - <<: Does not contain a valid number
>>+.<<: Does not contain a valid number
>>0001<<: 0001 (0001)
>>0..2<<: Does not contain a valid number
>>++1<<: Does not contain a valid number
>>A1.0B<<: Does not contain a valid number
>>A1<<: Does not contain a valid number
So our pattern correctly accepts the strings we intended to be recognised as numbers and rejects the others.


Let us turn to some other patterns now:

Finally: Regular expressions are very powerful, but they have certain theoretical limitations. One of these limitations is that they are not suitable for parsing arbitrarily nested text.

You can experiment with regular expressions using the VisualRegexp or Visual REGEXP applications.

More on the theoretical background and practical use of regular expressions (there is lots to cover!) can be found in the book Mastering Regular Expressions by J. Friedl.