Skip to content

Normalizr v0.1.6

Compare
Choose a tag to compare
@davidmogar davidmogar released this 09 May 13:48
· 206 commits to master since this release

Version 0.1.6 brings an improved way to execute a group of normalizations using a list. The next piece of code shows the same normalizations applied over the text "Who let the dog out?" using functions calls and list invocation:

normalizr = Normalizr(language='en')

# Without normalize function
text = 'Who    let   the dog out?'
text = normalizr.remove_extra_whitespaces(text)
text = normalizr.replace_punctuation(text, replacement=' ')
text = normalizr.remove_stop_words(text)
text = normalizr.remove_extra_whitespaces(text)

print(text)

# With normalize function
normalizations = [
        'remove_extra_whitespaces',
        ('replace_punctuation', {'replacement': ' '}),
        'remove_stop_words',
        'remove_extra_whitespaces'
]

print(normalizr.normalize('Who    let   the dog out?', normalizations))