Using regular expressions – Part 2: Finding and replacing text

Regular expressions (or regexes for short) are indispensible in find and replace operations. It doesn't matter how deeply buried the text you're looking for is, a well-defined regex will locate it in no time. Unfortunately, well-defined regexes are not always easy to create. Part 1 of this tutorial series covered the basic building blocks. In this second part, you'll put some of those principles to practical use through the Dreamweaver Find And Replace dialog box. You'll also learn how to use regexes in ColdFusion, JavaScript, and PHP.

The eye often deceives. Instead of seeing a repeated word, you read what you think should be there. You've likely seen something similar to Figure 1 many times, but the first time you saw it, you probably read it as Paris in the spring, and missed the second the. Once you're aware of the repeated word you can't miss it, but what about repeated words buried in dozens of lines of text?

The sample files for this tutorial contain a long extract from David Copperfield by Charles Dickens, in which four words have been deliberately duplicated. Searching for them visually would take a long time, but you can use a regex to find and delete them instantly.

Finding repeated words

Before you can delete the repeated words, you need to find them.

1. Open repeated_01.html from the sample files for this tutorial in Dreamweaver. It doesn't matter whether you're in Design view or Code view.
2. Choose Edit > Find And Replace to open the Find And Replace dialog box (you can also press Ctrl+F on Windows or Cmd+F on Mac OS X).
3. Select Use Regular Expression and deselect Match Case.
4. Select Current Document for the Find In option and select Source Code for the Search option.
5. Type the following regex in the Find text box:

This searches for a word boundary, followed by one or more word characters in a capturing group, another word boundary, a literal space, and a backreference to the capturing group. In other words, it looks for a standalone word, captures its value, and looks for it to be repeated immediately after a space.

6. Make sure the cursor is located at the start of the document, and click Find Next. Dreamweaver should highlight the the at the beginning of the third paragraph (see Figure 2).

7. Click Find Next again. It should detect and and at the end of the fourth paragraph.
8. Click Find Next twice again to find the remaining duplicate words (a a at the end of the seventh paragraph, and The the in the eleventh paragraph).

Note: The final duplicate is found because Dreamweaver treats regexes as case-insensitive. When using this regex with a programming language, you need to turn on the option for a case-insensitive match.

Deleting duplicate words

To eliminate the duplicate, you need to replace both words with just the first one. Within the regex, you use \1 as a backreference to the first capturing group. In a replace operation, the same value is represented by $1.

1. Keep the regex from the preceding exercise in the Find text box.
2. Type $1 in the Replace text box
3. Click Replace All.

The Search panel opens and reports the results. Each replacement text is underlined by a red wavy line (see Figure 3). All the duplicates are gone.

Making sure only complete words are deleted

It's too early to celebrate. Regexes have a nasty habit of doing things you don't expect.

1. Open repeated_02.html from the sample files.

   To make it easier to see what's happening, this file contains a single sentence with just one pair of repeated words (see Figure 4).

2. Run the same Find and Replace operation as in the preceding exercise.

   The Search panel reports that three replacements were made, not just one (see Figure 5).

The regex looked only for repeated characters, not complete words. As a result, The theme was shortened to Theme and his history to history.

3. Choose File > Revert, and click Yes to confirm that you want to undo the changes.
4. To ensure that the value matched by the backreference is a complete word, add a word boundary at the end of regex in the Find text box. It should look like this:

5. Click Replace All again. This time only the second was is deleted. Both The theme and his history remain intact.

Dreamweaver CS5 comes with 16 predefined CSS layouts, which you can access from the New Document dialog box (see Figure 6).

To help you understand what the style rules are for, the designer, Stephanie Sullivan, has sprinkled the style sheet liberally with comments—the one used in the next exercise has no fewer than 33. The comments are great for learning purposes, but you should delete them before the site goes live. It's time for another regex.

1. Open css_comments.html in the sample files for this tutorial. Alternatively, select the 2 Column Liquid, Right Sidebar, Header and Footer HTML layout in the New Document dialog box in Dreamweaver CS5.
2. Switch to Code view, and examine the CSS comments.

   The comments begin with /* and end with */. As explained in Part 1, * is a quantifier that means match 0 or more times. So, to match a literal asterisk, you need to escape it with a backslash.

3. Type the following regex in the Find text box of the Find And Replace dialog box:

The regex uses three asterisks. The two outer ones are escaped with backslashes, so they are treated as literal characters. The middle one acts as a quantifier for the dot metacharacter, which matches anything except a newline. So, this regex looks for /* and */ with anything (or nothing) in between.

Note: In many programming languages, the beginning and end of a regex is marked by characters known as delimiters. If your regex flavor uses forward slashes as a delimiter, you need to escape any literal forward slashes with a backslash. I cover this issue in more detail in Using regular expressions in ColdFusion, JavaScript, and PHP.

4. Make sure your cursor is at the top of the page in Code view, and click Find Next. Dreamweaver will highlight the first CSS comment (see Figure 7).

5. Click Find Next again. Dreamweaver will select the much longer CSS comment that begins on the following line.
6. Since the regex seems to be working, make sure the Replace text box is empty, and click Replace All.

   The Search panel should report that 32 replacements have been made.

7. Scroll down the page until just after the style rule for the header class.

   As you can see, a long, multiline comment has been left behind (see Figure 8). The dot metacharacter matches anything except a newline, so the regex failed when it encountered this comment.

Note: By default, Dreamweaver soft wraps code when it reaches the right edge of Code view, but doesn't insert a newline character. This is the only comment in the file that contains newline characters, as indicated by the line numbers on the left.

8. You need all the comments back before you can refine the regex, so select File > Revert, and click Yes when prompted to confirm you want to undo the changes.

   Most regex flavors have an option to make the dot metacharacter match a newline, but this is not supported in JavaScript, which Dreamweaver uses in the background for its Find and Replace operations. However, it's easy to create a character class that does the same.

   As I explained in Part 1, there are three pairs of metacharacters that match the opposite of each other. Put any of those pairs together in a character class, and it matches anything. For example, \w matches any word character, and \W matches anything that isn't a word character. So, [\w\W] matches anything, including a newline.

9. Replace the dot metacharacter with the character class like this:

10. Position your cursor at the top of the page, and click Find Next.

    This certainly copes with newlines, but it grabs everything from the beginning of the first CSS comment to the end of the last one. The * quantifier is greedy.

11. Change * into a lazy quantifier by adding a question mark after it like this:

12. Select File > Revert, and click Yes when prompted to confirm you want to undo the changes.
13. Move your cursor back to the top of the page, and click Find Next a couple of times to verify that a single comment is selected each time.
14. Click Replace All.

The Search panel should report there have been 33 replacements. This time, the multiline comment has been deleted, too.

The predefined CSS layouts in Dreamweaver CS5 use an image placeholder with an inline style for the logo. The instructions tell you how to remove the inline styles through the CSS Styles panel. You can use a regex instead.

1. Continue working with css_comments.html from the sample files. Make sure you are in Code view.

   HTML tags begin with < followed by the tag name, and end with >. The simple way to find a whole tag with a regex is to use <[^>]*>, which matches a pair of angle brackets with anything in between, as long as it's not a closing angle bracket.

2. Type <[^>]*> in the Find text box, and click Find Next.

   Depending on where your cursor is at the time, the next HTML tag will be highlighted (see Figure 9).

Note: This regex matches any HTML, XML, or ColdFusion tag. It works in the vast majority of cases, but fails if the value of an attribute contains a closing angle bracket (for example, as a greater than sign in a JavaScript event handler). Creating a regex that covers all possible combinations is complex and time-consuming. Sometimes, it is better to understand the limitations of a regex than to strive for 100% perfection.

3. Refine the regex by replacing the closing angle bracket with style= like this:

4. Click Find Next.

   This selects the <img> tag for the logo placeholder up to and including the style attribute and the equal sign (see Figure 10).

You want to delete the style attribute, so you need to create a capturing group to preserve everything up to that point by adding parentheses like this:

(<[^>]*)style=

Dreamweaver produces clean code, so you know that the style attribute is followed by an equal sign, and the value is enclosed in double quotes. But what if the code has come from elsewhere? There might be spaces around the equal sign, and the value might be in single quotes.

To match zero or more spaces you need \s*.

To match single or double quotes, you need the alternation metacharacter. You also need to use a capturing group so you have a backreference to the closing quote.

5. Refine the regex like this:

This surrounds the equal sign on both sides with zero or more spaces. Then a second capturing group alternates between single and double quotes.

Next comes a negative character class that uses the second backreference—in other words, it looks for anything except whatever was matched in the second capturing group (either a single or double quote). Contrary to what you might expect, the quantifier applied to this negative character class needs to be lazy (*?). If you use a greedy quantifier, such as * or +, the regex will grab everything in the rest of the code up to the final double quote.

Finally, the last \2 looks for the value stored in the second backreference (again either a single or double quote).

6. Test the regex again.

   You should now have everything up to and including the closing quote of the style attribute (see Figure 11).

7. To complete the regex, you need to get everything that follows the style attribute up to and including the closing angle bracket. In the current example, all that's left are the closing slash and the angle bracket, but in another case, you might have other attributes. Still, it's easy to accomplish with a negative character class and the closing bracket. The final section also needs to be in a capturing group so you can use it in the replace operation.
8. Refine the regex further like this:

After all that, the values you need for the Replace text box are remarkably simple. All that's necessary are the values in the first and third backreferences.

9. Type $1$3 in the Replace text box.
10. Click Find Next, and verify that the whole <img> tag has been selected.
11. Click Replace. The inline style should have been removed, leaving the other attributes intact.

Admittedly, this is a lot of work to get rid of a single inline style. But what if you need to get rid of inline styles in dozens of pages? The same regex does the job, and it never gets tired.

Part 1 of this tutorial showed how to match North American phone numbers with the following regex:

This makes optional both the parentheses around the area code and the hyphen between the last two sets of digits. A consistent format looks much smarter. Applying one is easy to do with capturing groups around each set of digits.

1. Open phone_numbers.html in the sample files.
2. Open the Find And Replace dialog box, and make sure Use Regular Expression is selected. It doesn't matter whether you're in Code view or Design view.
3. Type the following regex in the Find text box:

This is the regex you used in Part 1 with capturing group parentheses added around each group of digits.

4. Type ($1) $2-$3 in the Replace text box (see Figure 12).

5. Click Replace All.

   The phone numbers will be formatted with parentheses around the area code and a hyphen between the last two sets of digits (see Figure 13).

This tutorial series has concentrated on building and testing regexes in Dreamweaver, but regular expressions are also widely used in programming languages. This section highlights a few things you need to be aware of when using a regex in ColdFusion, JavaScript, or PHP.

ColdFusion

The most important differences in the regex flavor supported by ColdFusion are as follows:

• The dot metacharacter always matches a newline.
• In a replacement string, backreferences use a backslash instead of a dollar sign; for example, use \1 instead of $1.
• To make ^ and $ match the beginning or end of a line, you must precede the regex with (?m). However, the text must use only a line feed character. You need to remove carriage returns from text created on Windows, and replace Mac carriage returns with line feeds. The parentheses of (?m) are not counted as a capturing group.

For more information see Using Regular Expressions in Functions in the ColdFusion help files.

JavaScript

There are two ways of creating a regex in JavaScript: using literal syntax or the RegExp() constructor.

Using literal syntax

To use literal syntax, wrap the regex in forward slashes, and assign it to a variable like this:

The forward slashes are delimiters indicating that everything between them should be treated as a regex. If the regex needs to match a literal forward slash, escape it with a backslash. For example, the regex for a CSS comment needs to be rewritten like this:

Using the RegExp() constructor

The RegExp() constructor requires the regex without delimiters as a string in single or double quotes. This avoids the need to escape forward slashes, but introduces a different complication, which is arguably worse. All backslashes need to be escaped. For example, instead of using \d and \w, you need to use \\d and \\w. The CSS comment regex looks like this:

Modifying a regex with flags

You can modify a regex by specifying one or more flags. JavaScript supports the three flags listed in Table 1.

Table 1. Flags supported by JavaScript to modify a regex's behavior

When using literal syntax, add the flag(s) after the closing delimiter like this:

When using the RegExp() constructor, pass the flags in a string as the second argument like this:

PHP

PHP supports both Perl-style and POSIX regular expressions. However, support for POSIX is deprecated, and will be removed in PHP 6. Use only Perl-style regexes with the PHP functions that begin with preg_. Scripts that use split() or functions that begin with ereg should be updated as soon as possible.

Regexes in PHP should be enclosed in delimiters, and then wrapped in quotes. The normal convention is to use forward slashes. However, if the regex contains literal forward slashes, you need to escape them with backslashes like this:

Because this can make a regex difficult to read, you can use any pair of nonalphanumeric characters as delimiters, as long as they're not also used inside the regex. This means you can rewrite the previous example like this:

You can also use curly braces ({}) or angle brackets (<>) as delimiters, again as long as they're not used as literal characters in the regex.

PHP regexes can be modified by adding one or more flags between the closing delimiter and quote. The most important flags are listed in Table 2.

Table 2. Flags used in PHP to modify a regex's behavior

Where to go from here

Creating regular expressions is fascinating, but it can also be frustrating. Part of the complexity lies in the different meanings some symbols (such as ^ and $) have depending on the context in which they're used. You also need to test your regexes thoroughly to make sure they don't match more than you expected—or don't match at all. Then there's the problem of the different flavors; PHP and ColdFusion support more features than JavaScript, but there are differences between each of them.

Fortunately, you're not alone. There are online resources and books that help you with regular expressions. Although you might enjoy the challenge of building your own regular expressions, it's not always necessary to reinvent the wheel. For many tasks, tested and proven solutions already exist. Just copy the relevant regex into the Dreamweaver Find And Replace dialog box or your programming language, and away you go. The following list is by no means exhaustive, but points you to some of the best resources:

• Regular-Expressions.info
• RegExLib.com
• Jan Goyvaerts & Steven Levithan, Regular Expressions Cookbook
• Jeffrey Friedl, Mastering Regular Expressions, Second Edition