PyThaiNLP v2.3.1

[wannaphong]

PyThaiNLP v2.3.1 is This release is a bug fix release of PyThaiNLP 2.3.

Bug Fixed

• Fix gensim Thai2Fit: Word2Vec error with Gensim 4.0 #546

Documentation: https://pythainlp.github.io/docs/2.3/index.html
Report bug: https://github.com/PyThaiNLP/pythainlp/issues

You can install or upgrade using pip install -U pythainlp

See PyThaiNLP 2.3 change log #445

Deprecation and other API changes

• NER change a ThaiNER model (from ThaiNER 1.4 to ThaiNER 1.5). If you need use ThaiNER 1.4 model, You can use version in ThaiNameTagger class. pythainlp.tag.named_entity.ThaiNameTagger(version: str = '1.4') (Docs: https://pythainlp.github.io/dev-docs/api/tag.html#pythainlp.tag.named_entity.ThaiNameTagger)

Tokenizer

• Add model option to attacut.tokenize() #484 Add: model option for attacut.tokenize()
• Add create_wordlist #502 Add: corpus.util.revise_wordset() to revise tokenization dictionary
• Add NERCut tokenization engine #503 Add: NERCut tokenization engine

Corpus

• License change:
  • All corpora, datasets, and documentation created by PyThaiNLP project are now released under Creative Commons Zero 1.0 Universal Public Domain Dedication License (CC0).
  • All language models created by PyThaiNLP project are released under Creative Commons Attribution 4.0 International Public License (CC-by).
• Remove instances with [ or ] from etcc.txt #449 Fix: remove instances with [ or ] from etcc.txt
• corpus.common.provinces() with details option #467 Add: corpus.common.provinces() can now return romanized names
• Add family names #476 Add: thai_family_names() to get a set of Thai family names
• Fix: Not found thailand_provinces_th.csv #486 #487 Fix: thailand_provinces_th.csv not found issue
• Update tag.rst #492 Fix: remove erroneous AITT tag from ORCHID to UD table -- thanks @c4n for the fix

POS Tagger

• Add LST20 Part-Of-Speech tagger model #464 Add: LST20 language model for part-of-speech tagging
• perception/orchid_ud pos tagger model still require nltk to load #468 Add: port PerceptronTagger from NTLK. POS tagging no longer needs NLTK for dependency.
• Update ORCHID POS tags Docs #478 Update: ORCHID POS tags documentation

Name Entity Tagging

• Update ThaiNER 1.4 to ThaiNER 1.5 #526 Update ThaiNER 1.4 to ThaiNER 1.5
• Add ThaiNameTagger version #538 Add ThaiNameTagger version and add ThaiNER 1.4 support

Transliterate

• Romanize failed in some examples #485 Fixed Romanize failed in some examples
• Add Thai W2P #511 Add Thai W2P (Thai Word-to-Phoneme converter)

Text Summarize

• Add mT5 text summarize #523 Add mT5 text summarize to pythainlp.summarize

Chunk parser

• Add pythainlp.tag.chunk #524 Add pythainlp.tag.chunk

Util

• Fix remove_repeat_vowels() bug that remove spaces between vowel #481 Fix: remove_repeat_vowels() bug that remove spaces between different vowels
• Add method to remove a word from trie #483 Add: add remove() method to remove a word from a trie -- thanks @korakot
• thai_strftime: Normalize output for unsupported directive #490 Fix: thai_strftime() - normalize output for unsupported directive (running in glibc and musl should produce the same output)
• Add emoji convert #512 Add: emoji_to_thai() to convert emoji to Thai description -- thanks @ppirch for the development
• Add thai keyboard distance #513 Add: thai_keyboard_dist() to calculate euclidean distance between two characters according to their location on a Thai keyboard layout -- thanks @ppirch for the development

Thanks all the contributors. (Image made with contributors-img)

We build Thai NLP.

PyThaiNLP