Jump to content

How to match whole words with a regular expression

+ 2
  Jan Goyvaerts's Photo
Posted Sep 16 2009 03:20 PM

Create a regex that matches cat in My cat is brown, but not in category or bobcat. Create another regex that matches cat in staccato, but not in any of the three previous subject strings.

Here are sample solutions for the various flavors:

Word boundaries


\bcat\b
Regex options: None
Regex flavors: .NET, Java, Javascript, PCRE, Perl, Python, Ruby

Nonboundaries


\Bcat\B
Regex options: None
Regex flavors: .NET, Java, Javascript, PCRE, Perl, Python, Ruby

Word boundaries

The regular expression token \b is called a word boundary. It matches at the start or the end of a word. By itself, it results in a zero-length match. \b is an anchor, just like the tokens introduced in the previous section.

Strictly speaking, \b matches in these three positions:

  • Before the first character in the subject, if the first character is a word character

  • After the last character in the subject, if the last character is a word character

  • Between two characters in the subject, where one is a word character and the other is not a word character

None of the flavors discussed in this book have separate tokens for matching only before or only after a word. Unless you wanted to create a regex that consists of nothing but a word boundary, these aren’t needed. The tokens before or after the \b in your regular expression will determine where \b can match. The \b in \bx and !\b could match only at the start of a word. The \b in x\b and \b! could match only at the end of a word. x\bx and !\b! can never match anywhere.

To run a “whole words only” search using a regular expression, simply place the word between two word boundaries, as we did with \bcat\b. The first \b requires the c to occur at the very start of the string, or after a nonword character. The second \b requires the t to occur at the very end of the string, or before a nonword character.

Line break characters are nonword characters. \b will match after a line break if the line break is immediately followed by a word character. It will also match before a line break immediately preceded by a word character. So a word that occupies a whole line by itself will be found by a “whole words only” search. \b is unaffected by “multiline” mode or (?m), which is one of the reasons why this book refers to “multiline” mode as “^ and $ match at line breaks” mode.


Nonboundaries

\B matches at every position in the subject text where \b does not match. \B matches at every position that is not at the start or end of a word.

Strictly speaking, \B matches in these five positions:

  • Before the first character in the subject, if the first character is not a word character

  • After the last character in the subject, if the last character is not a word character

  • Between two word characters

  • Between two nonword characters

  • The empty string

\Bcat\B matches cat in staccato, but not in My cat is brown, category, or bobcat.

To do the opposite of a “whole words only” search (i.e., excluding My cat is brown and including staccato, category, and bobcat), you need to use alternation to combine \Bcat and cat\B into \Bcat|cat\B. \Bcat matches cat in staccato and bobcat. cat\B matches cat in category (and staccato if \Bcat hadn’t already taken care of that).


Word Characters


All this talk about word boundaries, but no talk about what a word character is. A word character is a character that can occur as part of a word.

Although all the flavors in this book support \b and \B, they differ in which characters are word characters.

.NET, Javascript, PCRE, Perl, Python, and Ruby have \b match between two characters where one is matched by \w and the other by \W. \B always matches between two characters where both are matched by \w or \W.

Javascript, PCRE, and Ruby view only ASCII characters as word characters. \w is identical to [a-zA-Z0-9_]. With these flavors, you can do a “whole words only” search on words in languages that use only the letters A to Z without diacritics, such as English. But these flavors cannot do “whole words only” searches on words in other languages, such as Spanish or Russian.

.NET and Perl treat letters and digits from all scripts as word characters. With these flavors, you can do a “whole words only” search on words in any language, including those that don’t use the Latin alphabet.

Python gives you an option. Non-ASCII characters are included only if you pass the UNICODE or U flag when creating the regex. This flag affects both \b and \w equally.

Java behaves inconsistently. \w matches only ASCII characters. But \b is Unicode-enabled, supporting any script. In Java, \b\w\b matches a single English letter, digit, or underscore that does not occur as part of a word in any language. \bкошка\b will correctly match the Russian word for cat, because \b supports Unicode. But \w+ will not match any Russian word, because \w is ASCII-only.

Regular Exp<b></b>ressions Cookbook

Learn more about this topic from Regular Expressions Cookbook.

This cookbook provides more than 100 recipes to help you crunch data and manipulate text with regular expressions. With recipes for popular programming languages such as C#, Java, Javascript, Perl, PHP, Python, Ruby, and VB.NET, Regular Expressions Cookbook will help you learn powerful new tricks, avoid language-specific gotchas, and save valuable time with this library of proven solutions to difficult, real-world problems.

See what you'll learn


Tags:
0 Subscribe


0 Replies