REMOVE_STOP_WORDS
Syntax
REMOVE_STOP_WORDS(<list>)
Description
Removes StopWords (common words) from a list of tokens. You can tokenize your text by using the EXTRACT_WORDS function.
Example
REMOVE_STOP_WORDS(#TokenizedText_Column)
| Tokenized text column | REMOVE_STOP_WORDS returns |
|---|---|
| [Unhappily, for, his, master, as, well, as, himself, his, curiosity, drew, him, unconsciously, farther, off, than, he, intended, to, go, At, last, having, seen, the, Parsee, carnival, wind, away, in, the, distance, he, was, turning, his, steps, towards, the, station, when, he, happened, to, espy, the, splendid, pagoda, on, Malabar, Hill, and, was, seized, with, an, irresistible, desire, to, see, its, interior, He, was, quite, ignorant, that, it, is, forbidden, to, Christians, to, enter, certain, Indian, temples, and, that, even, the, faithful, must, not, go, in, without, first, leaving, their, shoes, outside, the, door, It, may, be, said, here, that, the, wise, policy, of, the, British, Government, severely, punishes, a, disregard, of, the, practices, of, the, native, religions] | [Unhappily, master, well, himself, curiosity, drew, unconsciously, farther, intended, go, last, having, seen, Parsee, carnival, wind, away, distance, turning, steps, towards, station, happened, espy, splendid, pagoda, Malabar, Hill, seized, irresistible, desire, see, interior, quite, ignorant, forbidden, Christians, enter, certain, Indian, temples, even, faithful, go, without, first, leaving, shoes, outside, door, here, wise, policy, British, Government, severely, punishes, disregard, practices, native, religions] |
| [Passepartout, however, thinking, no, harm, went, in, like, a, simple, tourist, and, was, soon, lost, in, admiration, of, the, splendid, Brahmin, ornamentation, which, everywhere, met, his, eyes, when, of, a, sudden, he, found, himself, sprawling, on, the, sacred, flagging, He, looked, up, to, behold, three, enraged, priests, who, forthwith, fell, upon, him, tore, off, his, shoes, and, began, to, beat, him, with, loud, savage, exclamations, The, agile, Frenchman, was, soon, upon, his, feet, again, and, lost, no, time, in, knocking, down, two, of, his, long-gowned, adversaries, with, his, fists, and, a, vigorous, application, of, his, toes, then, rushing, out, of, the, pagoda, as, fast, as, his, legs, could, carry, him, he, soon, escaped, the, third, priest, by, mingling, with, the, crowd, in, the, streets] | [Passepartout, thinking, harm, went, simple, tourist, soon, lost, admiration, splendid, Brahmin, ornamentation, everywhere, met, eyes, sudden, found, himself, sprawling, sacred, flagging, looked, up, behold, three, enraged, priests, forthwith, fell, upon, tore, shoes, began, beat, loud, savage, exclamations, agile, Frenchman, soon, upon, feet, again, lost, time, knocking, down, two, long-gowned, adversaries, fists, vigorous, application, toes, rushing, out, pagoda, fast, legs, carry, soon, escaped, third, priest, mingling, crowd, streets] |