Tokenize repeating dots and commas from numbers (fix #461)

[wannaphong]

What does this changes

I change a rule of _RE_NONTHAI in newmm and multi_cut.

How this fixes it

Add rule of float number and number.

Fixes #461

Your checklist for this pull request

🚨Please review the guidelines for contributing to this repository.

• Passed code styles and structures
• Passed code linting checks and unit test

[coveralls]

Coverage increased (+0.05%) to 95.794% when pulling 86eae1c on fix-461 into c7a37b2 on dev.

[wannaphong]

Colab: https://colab.research.google.com/drive/1OcC7mCwAQQvx7qC9o6DdXcre7V4ohCtp?usp=sharing

[pep8speaks]

Hello @wannaphong! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-05-20 09:14:55 UTC

[bact]

Nice work. Can we add test cases for this?

May be something like:

• "19..." -> ['19', '...']
• "19." -> ['19', '.']
• "19.84" -> ['19.84']
• "127.0.0.1" -> ['127.0.0.1']
• "USD1,984.42" -> ['USD', '1,984.42']

[wannaphong]

Nice work. Can we add test cases for this?

May be something like:

* "19..." -> ['19', '...']

* "19." -> ['19', '.']

* "19.84" -> ['19.84']

* "127.0.0.1" -> ['127.0.0.1']

* "USD1,984.42" -> ['USD', '1,984.42']

OK

[bact]

I think the code and the test cases themselves are good (thanks), but the problem is around coverall for macOS hmm..

[bact]

The code and the test cases are good. Thank you.