Porter Stemmer in C: How the Porter Stemming Algorithm Works, Step by Step

Run 42,603 English words through the Porter stemmer and they collapse to 25,749 stems: connect, connected, connecting, connection and connections all become connect. Yet three widely used versions of the “same” stemmer do not agree with each other. The 1980 paper and Martin Porter’s own C code give different stems for 67 of those words, and NLTK’s default PorterStemmer differs from both on hundreds more.

This guide explains how the Porter stemming algorithm works, from its definition of a consonant and the “measure” of a word to the five steps that strip suffixes, and implements it in C with a switch between the paper’s rules and Porter’s reference behaviour. It then checks the code word for word against 42,603 published test words in both modes, shows nine mistakes that a quick test misses, and measures the speed. Every program was compiled and run for this article on Ubuntu 24.04 (x86-64) with GCC 13.3 and Clang 18.1.3 under -std=c11 and -std=c17, with -Wall -Wextra -pedantic -Werror and no warnings; the tests also run clean under AddressSanitizer, UndefinedBehaviorSanitizer and Valgrind, and every output block is captured verbatim. The same checks were repeated under WSL, and the code builds and passes its tests with MSVC 19.44 on Windows at /W4 /WX; the code is in a GitHub repository whose workflow repeats them on each commit.

What Is the Porter Stemmer?

The Porter stemmer is an algorithm that reduces English words to a common stem by removing suffixes, so that related forms such as connect, connected and connection map to the same key. It applies about 60 suffix rules in five steps, each guarded by a simple condition on the length of the remaining stem. Martin Porter published it in 1980, and it became the standard stemmer for English search engines.

Porter wrote the algorithm in 1979 at the Computer Laboratory in Cambridge, England, and published it as “An algorithm for suffix stripping” in Program, volume 14, number 3, pages 130 to 137 (DOI 10.1108/eb046814). His first implementation was in BCPL. The Porter Stemming Algorithm page he maintains lists implementations in many languages, records three small differences between his own code and the published paper, and recommends the later Snowball English stemmer, also called Porter2, for practical work. The original remains the one to use when results must be repeatable exactly.

A stemmer is not a dictionary. Its output is a key for matching, not a word for display:

Search engines stem both the documents and the query, so studying finds studies even though studi is not a word anyone would type.

How the Porter Stemming Algorithm Works

Consonants, Vowels and the Measure m

Every rule in the algorithm depends on one idea: how much of a word is left in front of a suffix. Porter measures that with m, the number of vowel-consonant sequences in the stem. Writing C for a run of consonants and V for a run of vowels, any word has the form C^m[V], so tree has m = 0, trouble has m = 1 and troubles has m = 2.

The only subtle letter is y. It counts as a consonant at the start of a word or after a vowel, as in yes and toy, and as a vowel after a consonant, as in sky and syzygy:

/* A consonant is a letter other than a, e, i, o, u, and other than a y
 * that follows a consonant. So y is a consonant in "yes" and "toy" and a
 * vowel in "sky" and "syzygy". */
static bool is_consonant(const char *b, size_t i)
{
    switch (b[i]) {
    case 'a': case 'e': case 'i': case 'o': case 'u':
        return false;
    case 'y':
        return i == 0 || !is_consonant(b, i - 1);
    default:
        return true;
    }
}

unsigned porter_measure(const char *b, size_t n)
{
    unsigned m = 0;
    size_t i = 0;
    while (i < n && is_consonant(b, i))      /* optional leading consonants */
        ++i;
    while (i < n) {
        while (i < n && !is_consonant(b, i)) /* a vowel sequence ...        */
            ++i;
        if (i == n)
            break;
        while (i < n && is_consonant(b, i))  /* ... closed by consonants    */
            ++i;
        ++m;
    }
    return m;
}

A suffix is removed only if the stem in front of it is long enough. Without that check, rate would lose its -ate and become r. With it, -ate goes only when m > 1, so activate becomes activ and rate stays whole.

The Five Steps at a Glance

The examples show what each step alone does to its input; a whole word passes through every step in turn.

Step 1: Plurals, -ed and -ing

/* Step 1a: plurals. caresses -> caress, ponies -> poni, cats -> cat. */
static void step1a(word *w)
{
    if (ends_with(w, "sses"))
        truncate_to(w, w->k - 2);
    else if (ends_with(w, "ies"))
        replace_suffix(w, "i");
    else if (ends_with(w, "ss"))
        ;                                   /* caress stays caress */
    else if (ends_with(w, "s"))
        truncate_to(w, w->k - 1);
}

/* Step 1b: -ed and -ing. agreed -> agree, plastered -> plaster,
 * hopping -> hop, filing -> file. */
static void step1b(word *w)
{
    if (ends_with(w, "eed")) {
        if (porter_measure(w->b, w->j) > 0)
            truncate_to(w, w->k - 1);       /* agreed -> agree; feed stays */
        return;
    }
    if (!((ends_with(w, "ed") || ends_with(w, "ing")) && has_vowel(w->b, w->j)))
        return;                             /* sing and bed are left alone */
    truncate_to(w, w->j);

    /* Tidy up the stem the suffix left behind. */
    if (ends_with(w, "at") || ends_with(w, "bl") || ends_with(w, "iz")) {
        w->b[w->k] = 'e';                   /* conflat -> conflate */
        truncate_to(w, w->k + 1);
    } else if (ends_double_consonant(w->b, w->k)) {
        char c = w->b[w->k - 1];
        if (c != 'l' && c != 's' && c != 'z')
            truncate_to(w, w->k - 1);       /* hopp -> hop; fall stays */
    } else if (porter_measure(w->b, w->k) == 1 && ends_cvc(w->b, w->k)) {
        w->b[w->k] = 'e';                   /* fil -> file */
        truncate_to(w, w->k + 1);
    }
}

/* Step 1c: a final y becomes i when the stem has a vowel. happy -> happi. */
static void step1c(word *w)
{
    if (ends_with(w, "y") && has_vowel(w->b, w->j))
        w->b[w->k - 1] = 'i';
}

The helpers these steps call, has_vowel, ends_double_consonant, ends_cvc for the *o test and ends_with, are a few lines each in the full source file. Step 1b does the most work. After it removes -ed or -ing, the stem is often damaged: conflated leaves conflat, hopping leaves hopp, filing leaves fil. Three repairs follow, in order: restore an e after -at, -bl or -iz; undo a doubled final consonant except l, s and z; and add an e to a short stem that ends consonant-vowel-consonant, the test the paper calls *o. Without these repairs conflated and conflate would get different stems.

Steps 2 to 4: The Longest Suffix Wins

Steps 2, 3 and 4 each hold a list of suffixes. The rule that applies is the one with the longest matching suffix, and if its condition fails, the step does nothing. It does not go on to try a shorter suffix:

typedef struct {
    const char *suffix, *replacement;
    signed char only;    /* -1 every mode, else only this porter_mode */
} rule;

/* Steps 2 to 4 apply the rule with the LONGEST matching suffix, and only
 * that one: if its condition fails, the step does nothing. */
static void apply_longest(word *w, const rule *rules, size_t count, unsigned min_m)
{
    const rule *best = NULL;
    size_t best_len = 0;
    for (size_t i = 0; i < count; ++i) {
        size_t n = strlen(rules[i].suffix);
        if ((rules[i].only < 0 || rules[i].only == (int)w->mode) && n > best_len &&
            ends_with(w, rules[i].suffix)) {
            best = &rules[i];
            best_len = n;
        }
    }
    if (best == NULL)
        return;
    w->j = w->k - best_len;
    if (strcmp(best->suffix, "ion") == 0 && /* step 4: -ion needs s or t */
        (w->j == 0 || (w->b[w->j - 1] != 's' && w->b[w->j - 1] != 't')))
        return;
    if (porter_measure(w->b, w->j) > min_m)
        replace_suffix(w, best->replacement);
}

The rule tables follow the paper, with two entries limited to one mode, which the next section explains:

/* Step 2: double suffixes to single ones, if m > 0. */
static const rule step2_rules[] = {
    {"ational", "ate", -1}, {"tional", "tion", -1}, {"enci", "ence", -1},
    {"anci", "ance", -1},   {"izer", "ize", -1},    {"abli", "able", PORTER_PAPER},
    {"bli", "ble", PORTER_STANDARD},                {"alli", "al", -1},
    {"entli", "ent", -1},   {"eli", "e", -1},       {"ousli", "ous", -1},
    {"ization", "ize", -1}, {"ation", "ate", -1},   {"ator", "ate", -1},
    {"alism", "al", -1},    {"iveness", "ive", -1}, {"fulness", "ful", -1},
    {"ousness", "ous", -1}, {"aliti", "al", -1},    {"iviti", "ive", -1},
    {"biliti", "ble", -1},  {"logi", "log", PORTER_STANDARD},
};

/* Step 3: -ic-, -full, -ness and friends, if m > 0. */
static const rule step3_rules[] = {
    {"icate", "ic", -1}, {"ative", "", -1}, {"alize", "al", -1}, {"iciti", "ic", -1},
    {"ical", "ic", -1},  {"ful", "", -1},   {"ness", "", -1},
};

/* Step 4: remove the suffix outright, if m > 1. */
static const rule step4_rules[] = {
    {"al", "", -1},   {"ance", "", -1}, {"ence", "", -1}, {"er", "", -1},
    {"ic", "", -1},   {"able", "", -1}, {"ible", "", -1}, {"ant", "", -1},
    {"ement", "", -1}, {"ment", "", -1}, {"ent", "", -1}, {"ion", "", -1},
    {"ou", "", -1},   {"ism", "", -1},  {"ate", "", -1},  {"iti", "", -1},
    {"ous", "", -1},  {"ive", "", -1},  {"ize", "", -1},
};

Step 5: Tidying the End

/* Step 5a: a final e goes if m > 1, or if m = 1 and the stem is not *o.
 * probate -> probat, cease -> ceas, but rate stays rate. */
static void step5a(word *w)
{
    if (!ends_with(w, "e"))
        return;
    unsigned m = porter_measure(w->b, w->j);
    if (m > 1 || (m == 1 && !ends_cvc(w->b, w->j)))
        truncate_to(w, w->j);
}

/* Step 5b: -ll becomes -l if m > 1. controll -> control, roll stays. */
static void step5b(word *w)
{
    if (w->k >= 2 && w->b[w->k - 1] == 'l' && ends_double_consonant(w->b, w->k) &&
        porter_measure(w->b, w->k) > 1)
        truncate_to(w, w->k - 1);
}

Step 5a keeps the e on short stems such as rate, because without it the stem would end consonant-vowel-consonant (rat) and collide with an unrelated word. Step 5b turns -ll into -l on long stems, so controlling and control meet at control.

Running All Eight Steps

size_t porter_stem_traced(char *b, enum porter_mode mode, porter_trace_fn trace, void *ctx)
{
    word w = {b, strlen(b), 0, mode};
    if (mode == PORTER_STANDARD && w.k <= 2)
        return w.k;                         /* "as", "is" and "us" stay whole */

    static const char *const names[] = {"1a", "1b", "1c", "2", "3", "4", "5a", "5b"};
    for (int step = 0; step < 8; ++step) {
        switch (step) {
        case 0: step1a(&w); break;
        case 1: step1b(&w); break;
        case 2: step1c(&w); break;
        case 3: apply_longest(&w, step2_rules, COUNT(step2_rules), 0); break;
        case 4: apply_longest(&w, step3_rules, COUNT(step3_rules), 0); break;
        case 5: apply_longest(&w, step4_rules, COUNT(step4_rules), 1); break;
        case 6: step5a(&w); break;
        default: step5b(&w); break;
        }
        if (trace != NULL)
            trace(names[step], w.b, ctx);
    }
    return w.k;
}

size_t porter_stem(char *word, enum porter_mode mode)
{
    return porter_stem_traced(word, mode, NULL, NULL);
}

porter_stem_traced calls an optional function after each step, which is how the stem --trace command shows a word’s progress:

generalizations
  step 1a  generalization
  step 1b  generalization
  step 1c  generalization
  step 2   generalize
  step 3   general
  step 4   gener
  step 5a  gener
  step 5b  gener
stem: gener

Four of the eight steps change generalizations, and each change was allowed because the stem left in front of the suffix was long enough: m(general) = 3 for step 2, m(gener) = 2 for steps 3 and 4.

Using the Porter Stemmer in C

The header has two functions that matter: porter_stem() stems a lowercase, NUL-terminated word in place and returns its new length, and porter_measure() exposes m for teaching and tests. This program stems a few word families:

/* stem_demo.c - stems a few groups of related words, then shows the three
 * words where Martin Porter's own implementation and the 1980 paper differ.
 * Build: gcc -std=c11 -Wall -Wextra -pedantic stem_demo.c porter.c -o stem_demo */
#include <stdio.h>
#include <string.h>

#include "porter.h"

static void show(const char *const words[], size_t count)
{
    for (size_t i = 0; i < count; ++i) {
        char buf[64];
        strcpy(buf, words[i]);
        porter_stem(buf, PORTER_STANDARD);
        printf("  %-16s -> %s\n", words[i], buf);
    }
}

int main(void)
{
    static const char *const connect[] = {"connect", "connected", "connecting",
                                          "connection", "connections"};
    static const char *const others[] = {"caresses", "ponies", "hopping", "filing",
                                         "happy", "relational", "generalizations",
                                         "probate", "controlling", "university",
                                         "universe"};
    puts("One stem for related words:");
    show(connect, sizeof connect / sizeof connect[0]);
    puts("Stems are keys, not dictionary words:");
    show(others, sizeof others / sizeof others[0]);

    puts("Where Porter's code and the 1980 paper differ:");
    static const char *const departures[] = {"as", "possibly", "archaeology"};
    for (size_t i = 0; i < 3; ++i) {
        char standard[64], paper[64];
        strcpy(standard, departures[i]);
        strcpy(paper, departures[i]);
        porter_stem(standard, PORTER_STANDARD);
        porter_stem(paper, PORTER_PAPER);
        printf("  %-16s standard: %-12s paper: %s\n", departures[i], standard, paper);
    }
    return 0;
}

Output:

One stem for related words:
  connect          -> connect
  connected        -> connect
  connecting       -> connect
  connection       -> connect
  connections      -> connect
Stems are keys, not dictionary words:
  caresses         -> caress
  ponies           -> poni
  hopping          -> hop
  filing           -> file
  happy            -> happi
  relational       -> relat
  generalizations  -> gener
  probate          -> probat
  controlling      -> control
  university       -> univers
  universe         -> univers
Where Porter's code and the 1980 paper differ:
  as               standard: as           paper: a
  possibly         standard: possibl      paper: possibli
  archaeology      standard: archaeolog   paper: archaeologi

The first group is the reason stemmers exist: five forms of one word share a key. The second shows what the keys look like. poni, happi and gener are not words, and university and universe share a stem although their meanings differ, an error called overstemming. The opposite error, understemming, leaves related words apart: the stemmer has no rule that brings absorb and absorption together.

To stem a file with one word per line, use the stem program: ./build/stem words.txt, or echo running | ./build/stem, which prints run. It converts each line to lowercase before stemming.

The 1980 Paper vs Porter’s Own Code: Three Differences

Porter’s reference implementation departs from the published paper in three places, each marked in his source code:

The code on this page implements both: PORTER_STANDARD follows Porter’s code and is the default, and PORTER_PAPER follows the paper exactly. Across the 42,603 test words, the two modes give different stems for 67 words. Tracing possibly in paper mode shows where the paper’s version stops short:

possibly
  step 1a  possibly
  step 1b  possibly
  step 1c  possibli
  step 2   possibli
  step 3   possibli
  step 4   possibli
  step 5a  possibli
  step 5b  possibli
stem: possibli

Step 1c turns the final y into i, and in paper mode step 2 has no rule for -bli, only for -abli, so possibli survives to the end. Porter describes his three changes as improvements. If you need results that match another system, find out which version it uses before you compare stems.

Checking a Porter Stemmer Against 42,603 Words

A stemmer that looks right on a dozen examples can still be wrong on hundreds of words, so this one is tested against published data. The vocabulary of 42,603 words and the paper’s expected stems come from the Snowball project’s test data. The expected stems for the standard mode were produced by running the same words through Martin Porter’s own ANSI C implementation. test_porter checks every word in both modes, plus the paper’s own examples of m and the edge cases:

42603 words: paper mode 0 wrong, standard mode 0 wrong
unit checks failed: 0

Agreement with expected output is strong evidence, but both reference files could share a mistake. As an independent check, tests/compare_nltk.py compares every stem with NLTK, a Python implementation written by different people, in each of its three modes:

NLTK 3.10.4 ORIGINAL_ALGORITHM vs --paper : 0 of 42603 differ
NLTK 3.10.4 MARTIN_EXTENSIONS  vs standard: 0 of 42603 differ
NLTK 3.10.4 NLTK_EXTENSIONS    vs standard: 785 of 42603 differ

NLTK’s ORIGINAL_ALGORITHM mode agrees with paper mode on all 42,603 words, and its MARTIN_EXTENSIONS mode agrees with the standard mode on all of them. NLTK’s default mode, NLTK_EXTENSIONS, adds further changes by NLTK contributors and gives 785 different stems: it keeps a final y after a vowel (plays → play, where Porter gives plai) and maps a few irregular forms directly (dying → die). That is a reasonable stemmer, but it is not the Porter algorithm, and two systems that both “use the Porter stemmer” can disagree because of it.

Nine Mistakes That Break a Porter Stemmer

A broken stemmer still produces plausible stems, so its mistakes are easy to miss. tests/mutations.py puts nine plausible mistakes into copies of porter.c, one at a time, builds each copy, and counts how many of the 42,603 stems change:

mistake                                  stems changed  example
y is always a consonant                             90  bygone: bygon -> bygone
y is always a vowel                                 59  abeyance: abey -> abeyanc
falls back to a shorter suffix                      89  agreement: agreement -> agreem
measures the whole word, not the stem             3745  abate: abat -> ab
step 4 needs m > 0 instead of m > 1               3737  abate: abat -> ab
-ion without the s or t condition                   44  annexions: annexion -> annex
*o forgets the w, x, y exception                   125  baying: bai -> baye
step 1b skips the tidy-up                         1292  abbreviated: abbrevi -> abbreviat
step 3 runs before step 2                          137  aesthetically: aesthet -> aesthetic
mistakes the vocabulary test would miss: 0

Every mistake changed at least one stem, so the vocabulary test catches all nine. Four of them are worth knowing by name:

• Getting y wrong. Treating y as always a consonant changed 90 stems; always a vowel, 59. Both pass any test that does not include a word like bygone or abeyance. Use the rule: y is a vowel only after a consonant.
• Falling back to a shorter suffix. In agreement, the longest suffix -ement fails its m > 1 condition, so step 4 must stop. Trying -ment and then -ent instead strips the word to agreem, and 89 stems change.
• Measuring the wrong part of the word. m must be computed on the stem in front of the suffix, not the whole word. Getting this wrong changed 3,745 stems, because the whole word always measures at least as long as its stem, so conditions pass that should fail: abate became ab.
• Running the steps in the wrong order. Swapping steps 2 and 3 changed 137 stems: aesthetically stopped at aesthetic instead of aesthet. Step 2 turns aestheticalli into aesthetical, which step 3’s -ical rule then shortens; run step 3 first, and it finds nothing to do.

In C there is one more trap: reading past the start of the word. Step 1 can leave a single letter (ied becomes i), so any code that looks at the next-to-last letter must check the length first. The tests include ied in both modes and run under AddressSanitizer, which reports a read before the start of a buffer as an error instead of returning whatever byte happens to be there.

Another mistake is not in the code at all: stemming a stem. The algorithm is not idempotent. Of the 42,603 test words, 1,752 have stems that change again if stemmed a second time: deliberations → deliber → delib. A search system that stems documents when it indexes them and then stems the stored stems again at query time will miss matches. Stem raw words exactly once, with the same stemmer and mode on both sides.

What Stemming Does to a Vocabulary

stem_stats runs the whole test vocabulary through the stemmer in the standard mode:

words:                      42603
changed by the stemmer:     27959 (65.6%)
letters removed:            17.8%
distinct stems:             25749
standard and paper differ:  67 words
stem changes if stemmed again: 1752 words

About two words in three change, and the vocabulary shrinks from 42,603 distinct words to 25,749 distinct stems, a reduction of 40% in the number of terms an index has to store. To count the distinct stems, the program sorts them with qsort and counts the changes between neighbours, the same approach the sorting algorithms comparison discusses. In a real search engine the stems become the keys of an inverted index, usually held in a hash table or a trie, each mapping to the documents that contain any word with that stem.

How Fast Is the Porter Stemmer in C?

How This Was Measured

All figures come from one machine in one session, single-threaded. No CPU pinning or frequency-scaling control was applied, and no percentiles were captured. The benchmark measures stemming words already split and lowercased, not reading or tokenizing text.

The C version stems a word in about a quarter of a microsecond, roughly 23 times faster than NLTK with the same rules, so stemming is rarely the slow part of indexing in either language. The C code is written for clarity rather than speed: it checks every rule in a step and recomputes strlen for each one, where Porter’s own code stores each suffix’s length and only checks the rules whose suffix ends in the right letter. For most uses that does not matter, because tokenizing, lowercasing and writing the index cost more.

When to Use Porter, Porter2 or a Lemmatizer

• Use the original Porter stemmer when you need results that match an existing index, a published experiment or another system that uses it. Porter has declared it frozen, which is exactly why it is still useful.
• Use the Snowball English stemmer (Porter2) for new English search systems. Martin Porter designed it as an improvement on the original, and implementations are available from the Snowball project for many languages.
• Use a lemmatizer when the output is shown to people or must be a real word, for example in a writing tool. It needs a dictionary and usually the part of speech, so it is slower and language-specific.
• Use no stemming for exact-match fields such as names, codes and identifiers, where Rate and rating must stay different.

Key Takeaways

• The Porter stemmer removes suffixes in five steps, each rule guarded by m, the number of vowel-consonant sequences in the stem in front of the suffix.
• In steps 2 to 4, the longest matching suffix decides, and a failed condition ends the step; falling back to a shorter suffix changed 89 of 42,603 stems.
• The paper and Porter’s own code differ in three rules, which change 67 of 42,603 stems; NLTK’s default mode differs from Porter’s code on 785.
• Test a stemmer against published data, not a handful of examples: all nine deliberate mistakes on this page changed stems that a short test would not have checked.
• Stems are keys, not words, and stemming is not idempotent: 1,752 stems change if stemmed again, so stem each raw word once.
• In C the stemmer handles about 4 million words per second on one core, about 23 times NLTK’s rate with the same rules.

Frequently Asked Questions

Conclusion

The Porter stemmer has lasted more than four decades because it does one job with very little machinery: a definition of a consonant, a way of measuring a stem, and about sixty rules applied in a fixed order. Writing it is a good exercise in reading a specification closely, since most of its bugs come from rules that are easy to misread, such as when y is a vowel, which part of the word m applies to, and what happens when the longest suffix’s condition fails.

It is also a reminder that “the same algorithm” is not always the same. The paper, Porter’s own code and NLTK’s default each give different stems for some words, and only a test against published output tells you which one you have. More algorithms built and tested the same way are in the algorithms section.

Source Code and Tests

All programs on this page are in the text-processing/porter-stemmer directory of the MYCPLUS C examples repository.

Build and test:

cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure

What the build checks. It compiles with GCC and Clang under C11 and C17 on Linux and with MSVC /W4 /WX on Windows, treating warnings as errors. CTest checks all 42,603 test words in both modes, the paper’s examples of m and the edge cases, that the demo, both traces and the statistics print exactly the outputs on this page, and that bad command lines are rejected. On Linux the workflow also runs everything under AddressSanitizer and UndefinedBehaviorSanitizer, checks that each of the nine mutations changes at least one stem, and compares every stem with NLTK 3.10.4. It does not run the benchmark, whose figures came from one machine.

What I Verified

The example directory contains run_all.sh for Ubuntu and WSL and run_all.ps1 for Windows, which run every check below and print one line per row.

Environments: Ubuntu 24.04.5 on a cloud VM; Ubuntu 24.04.5 under WSL2 (kernel 5.15) with GCC 13.3 and Clang 18.1.3; Windows build 26300 with Visual Studio 2022 17.14 (MSVC 19.44).

Not covered: 32-bit builds, compilers newer than GCC 13.3 and Clang 18.1.3, Clang’s sanitizers (they run only in CI), timing on any other machine, and non-ASCII input, which the stemmer treats as consonants.