Chapter 2: Learn

_PS1+ kirim_VSA gloss book recipe I I send english my recipe book I send output buku resep ku ku kirim Figure 4: MorphInd analysis examples for Indonesian phrases that contain cliticized word and the preprocessing output after separating the clitic(s). The Verb Phrase's clitics are the Subject or Object of the Verb, while the enclitic on the Noun Phrase is a Possessive Pronoun of the Noun. gual data are: • Europarl Corpus • News Commentary Corpus • News Crawl Corpus (2007-2011) We treat them as seven separate LMs, which correspond to seven LM features in MOSES configuration file. We use SRILM (Stolcke, 2002) to build the LMs. The quality of the translation result is measured using the BLEU score metric (Papineni et al., 2002). 5.2 unclitic system As we have seen in Figure 2, Indonesian clitics have a fairly simple pattern and each is aligned to a different individual word in English. We use a finite state Indonesian morphological analyzer tool, MorphInd (Larasati et al., 2011) to find the correct clitics instead of just using a simple pattern matching with regular expression. This is to make sure that we do not cut the word in a wrong morpheme segmentation. We preprocess the text by separating the clitics given the Indonesian clitics schema and MorphInd correct clitics detection, to make the alignment model simpler. Figure 4 shows several MorphInd analysis examples. The input shows the original words in Indonesian and the output shows the new text after we apply the preprocessing. The preprocessing is applied on the training, the tuning, and the testing data. Then we build another SMT system (unclitic) with the same setting as the baseline system but using the new preprocessed data. 6 Result and Discussion For this study, we make three combinations of dataset comparison (F-H, E-I-H, and F-S) to see how is the translation quality differs by using different datasets. Then we also observe the gain or loss caused by the preprocessing on the Indonesian clitics. The translation evaluation as a whole can be seen in Figure 5. 149 Figure 5: The baseline and unclitic SMT systems translation quality in terms of BLEU Score and their corresponding OOV Rate (%) on different datasets (F-H-S-E-I). 6.1 Working with Smaller Training Data (F-H) The Indonesian-English parallel data is relatively small to begin with (±45K sentences or ±1M words). Here we try to push it even further to train an SMT system with only half of the training data that we have and observe the effect of applying the preprocessing on the clitics. In this experiment, we compare the systems that are trained on F and H datasets, where the training data is in the same type but differ in size. Considering the small number of the training data that H has, having more data at this stage still helps to get a better translation quality. Here we also see that the smaller system gain more improvement by applying the preprocessing. 6.2 Different Text Categories (E-I-H) Here we compare three different systems trained on three different smaller training data (21K-24K sentences), i.e. E,I, and H datasets. Here we see that the E dataset has a very high Out-of-Vocabulary (OOV) rate, which makes a poor translation result, and even the clitic preprocessing cannot help to improve the translation. In spite of that, the system trained on H and I datasets gain a better translation quality by applying the preprocessing. 6.3 Translating Spoken Indonesian (F-S) Indonesian speakers tend to use more clitics in Indonesian spoken language, than in a formal written text. Here we put the focus on the spoken language by comparing system trained on S dataset (subtitles as the tuning and testing data) and compared it with system trained on F dataset (the mixed types text). The BLEU score for the baseline S is far below the baseline F, although their training data sizes only differ slightly (±43K (F) and ±42K (S) sentences). This happens because Indonesian spoken dialogue is more difficult to translate. In spite of the score difference, here we see that translating the subtitle text gains the most improvement by applying the clitic preprocessing. 7 Conclusion We showed one linguistically motivated example on how to incorporate morphological information into 150 an NLP application for Indonesian. We used the state-of-the art SMT tool, MOSES, and utilized the information provided from an Indonesian morphological analyzer, MorphInd. We compared five different SMT systems in three different combinations, where we also applied a preprocessing on the datasets. We saw that the preprocessing overall improves the translation quality, except on the E dataset (with en-to-id translated text as the training data) where its OOV rate is too high. The S (subtitle text) dataset benefited the most from the preprocessing. 8 Future Work There are still other straightforward Indonesian language constructions that can be exploited to improve Indonesian-English SMT system translation quality as part of a preprocessing. Moving a step further from morphology, incorporating additional syntactical information will be an interesting approach to do. For example, since Indonesian and English have an opposite dependency for the Noun Phrase head-modifier construction, preordering Indonesian words in a Noun Phrase before the translation takes place will be a good approach to improve the translation quality. Having more Indonesian-English parallel sentences for the training will hopefully improve the translation quality, since currently the parallel data is still very small. This will also increase the interest to do research in this language pair. Acknowledgments The research leading to these results has received funding from the European Commission's 7th Framework Program under grant agreement n ◦ 238405 (CLARA), by the grant LC536 Centrum Komputacnˇ ´ı Lingvistiky of the Czech Ministry of Education, and this work has been using language resources developed and/or stored and/or distributed by the LINDAT-Clarin project of the Ministry of Education of the Czech Republic (project LM2010013). References M. Adriani, J. Asian, B. Nazief, SMM Tahaghoghi, and H.E. Williams. 2007. Stemming indonesian: A confix-stripping approach. ACM Transactions on Asian Language Information Processing (TALIP), 6(4):1–33. T. Baldwin. 2006. Open source corpus analysis tools for malay. In In Proc. of the 5th International Conference on Language Resources and Evaluation. Citeseer. A. Bisazza and M. Federico. 2009. Morphological preprocessing for turkish to english statistical machine translation. In Proc. of the International Workshop on Spoken Language Translation, pages 129–135. BPPT. 2009. Final report on Statistical Machine Translation for Bahasa Indonesia - English and English - Bahasa Indonesia. Technical report, Badan Pengkajian dan Penerapan Teknologi. Nizar Habash and Fatiha Sadat. 2006. Arabic preprocessing schemes for statistical machine translation. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pages 49–52, New York City, USA, June. Association for Computational Linguistics. P. Koehn, F.J. Och, and D. Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 48–54. Association for Computational Linguistics. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pages 177– 180, Prague, Czech Republic, June. Association for Computational Linguistics. Septina Dian Larasati, Vladislav Kubon, and Dan Zeman. ˇ 2011. Indonesian morphology tool (MorphInd): Towards an indonesian corpus. Systems and Frameworks for Computational Morphology, pages 119–129, august. Septina Dian Larasati. 2012. IDENTIC corpus: Morphologically enriched indonesian-english parallel corpus. In Proceedings of the Eighth conference on International Language Resources and Evaluation (LREC'12), Istanbul, Turkey, may. European Language Resources Association (ELRA). M.P. Marcus, M.A. Marcinkiewicz, and B. Santorini. 1993. Building a large annotated corpus of english: The penn treebank. Computational linguistics, 19(2):313–330. Preslav Nakov and Hwee Tou Ng. 2009. Improved statistical machine translation for resource-poor languages 151 using related resource-rich languages. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1358–1367, Singapore, August. Association for Computational Linguistics. P. Nakov and H.T. Ng. 2011. Translating from morphologically complex languages: a paraphrase-based approach. In Proceedings of the Meeting of the Association for Computational Linguistics (ACL2011), Portland, Oregon, USA. F.J. Och and H. Ney. 2003. A systematic comparison of various statistical alignment models. Computational linguistics, 29(1):19–51. K. Papineni, S. Roukos, T. Ward, and W.J. Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. James Neil Sneddon. 1996. Indonesian Reference Grammar. Allen & Unwin. A. Stolcke. 2002. SRILM - an extensible language modeling toolkit. In Seventh International Conference on Spoken Language Processing. Reyyan Yeniterzi and Kemal Oflazer. 2010. Syntax-tomorphology mapping in factored phrase-based statistical machine translation from english to turkish. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 454–464, Uppsala, Sweden, July. Association for Computational Linguistics. 152