Bangla Interrogative Sentence Identification from Transliterated Bangla Sentences
Hamid, M. M., Alam, T., Ismail, S., & Rabbi, M. F.
International Conference on Bangla Speech and Language Processing (ICBSLP), IEEE · 2018
Huge numbers of Bangla questions are written online in transliterated (Latin-alphabet) form — often without question marks or question words, which makes them hard to identify automatically. We compared rule-based, classical machine learning, and deep learning approaches, reaching 92.11% accuracy with a Multilayer Perceptron and demonstrating CNNs as a viable model for Bangla NLP.
The problem
Identifying interrogative sentences matters for data analytics, chatbots, question-answering services, and smart-assist applications: service providers can learn about client expectations and queries from the questions users post. But in transliterated Bangla, question marks are frequently omitted (almost 30% of online questions lack one), a question mark doesn't always signal a question ("Tomar eto boro spordha?" expresses exclamation), and people often form questions without Bangla question words — making identification challenging.
What we did
We built corpora from social media and chat text: an interrogative corpus of 700 sentences (149 of which contain no Bangla question word at all), a cricket-domain corpus, and a mixed-domain corpus that adds university-admission chatbot data. We then explored three families of approaches: a rule-based approach with rules derived from grammar and question-word position analysis; supervised learning with Support Vector Machine, k-Nearest Neighbors, Multilayer Perceptron, and Logistic Regression; and a deep learning approach with a Convolutional Neural Network trained on the mixed-domain corpus (80/10/10 train/validation/test split). We released the datasets and code publicly.
Key results
- The rule-based approach proved unsatisfactory: sentence variations are too high for any concrete set of rules — e.g., a question word as the last word usually signals a question, but not always.
- Supervised learning performed strongly: SVM 91.43%, k-NN 75.98%, Multilayer Perceptron 92.11%, and Logistic Regression 91.68% accuracy.
- The CNN achieved 84.64% accuracy on the test set (85.77% on validation) and correctly classified extreme transliteration variants like "ajka kala kokhan" — establishing CNNs as a promising model for Bangla NLP.
- Accuracy dropped when training and test data came from different domains, highlighting domain dependence as a key challenge for transliterated Bangla.