Tamilinayavaani - integrating tva open-source spellchecker with python தமிழ் இணையக் கல்விக்கழகம் - பொதுக்கள அறிவிப்பு https://commons.wikimedia.org/wiki/File:Tamil-Virtual-Academy-Copyright-Declaration.jpg தமிழ் இணையக் கல்விக்கழகம் - தமிழ் கணிமைக் கருவிகள் http://www.tamilvu.org/ta/content/%E0%AE%A4%E0%AE%AE%E0%AE%BF%E0%AE%B4%E0%AF%8D%E0%AE%95%E0%AF%8D-%E0%AE%95%E0%AE%A3%E0%AE%BF%E0%AE%A9%E0%AE%BF%E0%AE%95%E0%AF%8D-%E0%AE%95%E0%AE%B0%E0%AF%81%E0%AE%B5%E0%AE%BF%E0%AE%95%E0%AE%B3%E0%AF%8D Open-Tamil Python library https://github.com/Ezhil-Language-Foundation/open-tamil Demo Applications on Open-Tamil Python library - http://tamilpesu.us Python port of TamilinaiyaVaani Spellchecker ://github.com/tshrinivasan/Tamilinaiya-Spellchecker Collected 1,53,548 unique tamil nouns https://github.com/KaniyamFoundation/all_tamil_nouns Collected 25,83,000 unique tamil words https://github.com/KaniyamFoundation/all_tamil_words கணியம் அறக்கட்டளையின் செயல்பாடுகள் https://github.com/KaniyamFoundation/ProjectIdeas/issues Series of blog posts on building an open source spellchecker for tamil - https://goinggnu.wordpress.com/category/spellchecker/

1. Tamil Internet Conference 2020
TamilInayaVaani - Integrating TVA Open-
Source spellchecker with Python
T. Shrinivasan, Nithya Duraisamy, Ashok Ramachandran, Manickkavasakam,
Arunmozhi, and A. Muthiah

2. Who are we?
Few Open Source Contributors from
Ezhil Foundation
Kaniyam Foundation
Thamizha
Mozilla Tamilnadu
Indian Linux Users group, Chennai
IndicNLP
Having similar dreams in many heads

3. Open source Tamil Spellchecker
A Dream for many years becoming real

4. Existing Efforts
●
Hunspell
●
GNU Aspell
●
LanguageTool.org
●
Open-Tamil Solthiruthi
●
Bloom Filter based spellchecker

5. Still long way to go
How long?

6. Problems with Tamil Spellchecker
●
Infinity Vocabulary
●
Rich in Morphology
●
Agglutinative
●
Free Word Order
●
Sandhi
●
...

7. Few Algorithms
Levenshtein distance search

8. Levenshtein distance search

9. Few Algorithms
Norvig Algorithm

10. Norvig Algorithm

11. Still not perfect
Research continues...

12. TamilinayaVaani
A Open Source Spellchecker from
Tamil Virtual Academy

15. TN Govt announcement
All the software released in GNU GPL V2
All digital content in CC-BY-SA

16. TamilinayaVaani
●
Developed as Desktop Version
●
C# based
●
Limited version of Vaani.neechalkaran.com
●
Cant use in Linux
●
Cant use as command line
●
Cant integrate with other applications

17. Porting to Python
Why?

18. Porting to Python
Python – Easy to develop further
Easy integration
Web applications
API
Scalable

20. Python Port Code
●
https://github.com/tshrinivasan/Tamilinaiya-Spellchecker

21. The beauty of Open Source
More Contributions

22. Open-Tamil Python Library
●
The defacto Python library for Tamil Computing
●
Process tamil text
●
Build Games, Tamil Utilities
●
http://Tamilpesu.us

24. Integrating with sandhichecker
●
Open-Tamil has a SandhiChecker
●
40+ rules
●
Added this sandhi Checker to Tamilinayavaani

25. Python Packaging
●
Easy install in any OS
●
Pip install tamiliyavaani

26. Sample Usage

27. Web Interface with TinyMCE
●
Added a good web interface

28. Web Interface

29. Web Interface

30. Web Interface

31. Web Interface

32. Web Interface

33. JavaScript
A JavaScript port is on the way

34. TODO
●
Provide API
●
Host as a Public website
●
Test and add more rules
●
Set edit distance=2
●
Find method to yield better alternate
●

35. Word Corpus
●
Collected 1,53,548 unique tamil nouns
●
Collected 25,83,000 unique tamil words
●
https://github.com/KaniyamFoundation/all_tamil_words
●
https://github.com/KaniyamFoundation/all_tamil_nouns

36. TODO
●
Clean them manually
●
Build a golden corpus for quick lookup
●
BloomFilter/SymSpell/LSTM and more

37. Please Contribute
●
Give Tamil Rules
●
Give Tamil Corpus
●
Write Code
●
Test
●
Document
●
Provide Hosting
●
Donate

38. Thanks
●
Muthu Annamalai
●
Tamilnadu Government
●
Neechalkaran
●
Nithya Duraisamy
●
Ashok Ramachandran
●
Manickkavasakam
●
Arunmozhi
●
And All Contributors for
Ezhil Foundation, Kaniyam
Foundation, Thamizha,
IndicNLP and all other
Open Source Teams

39. Contact
●
T Shrinivasan
●
tshrinivasan@gmail.com
●
Kaniyam.com