Rudolf Rosa, Ondřej Dušek, Michal Novák, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz Translation Model Interpolation for Domain Adaptation in TectoMT Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics DMTW, Praha, 3 September 2015
Why domain adaptation? not to do MT as Europarl science reasonably good for e.g. economic news not particularly good for IT helpdesk Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 2/29
Why domain adaptation? not to do MT as Europarl science reasonably good for e.g. economic news not particularly good for IT helpdesk Jak pošlu přílohu mailem? How do I send an attachment by e-mail? Click on the paperclip icon and select the file. Klikněte na ikonu sponky a vyberte soubor. Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 3/29
Why model interpolation? available parallel data out-of-domain (Europarl): 2 000 000 sentences in-domain (IT helpdesk): 1 000 sentences Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 4/29
Why model interpolation? available parallel data out-of-domain (Europarl): 2 000 000 sentences in-domain (IT helpdesk): 1 000 sentences train on out+in concatenation effect of in negligible (overpowered by out) Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 5/29
Why model interpolation? available parallel data out-of-domain (Europarl): 2 000 000 sentences in-domain (IT helpdesk): 1 000 sentences train on out+in concatenation effect of in negligible (overpowered by out) train on in only too weak, low coverage, worse than out alone Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 6/29
Why model interpolation? available parallel data out-of-domain (Europarl): 2 000 000 sentences in-domain (IT helpdesk): 1 000 sentences train on out+in concatenation effect of in negligible (overpowered by out) train on in only too weak, low coverage, worse than out alone train 2 translation models, one on in, one on out final TM = (weighted) interpolation of intm and outtm Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 7/29
Why TectoMT (deep transfer MT)? better meaning preservation better generalization from small data limited support for TM interpolation no support for tuning of TM weights Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 8/29
Why TectoMT (deep transfer MT)? better meaning preservation better generalization from small data limited support for TM interpolation solved no support for tuning of TM weights TODO Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 9/29
Outline Motivation TectoMT (deep transfer) Translation model interpolation in TectoMT Evaluation Conclusion Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 10/29
TectoMT deep transfer Jak pošlu přílohu mailem? Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 11/29
TectoMT deep transfer Jak pošlu přílohu mailem? poslat Jak adv #PersPron drop příloha n:4 mail n:7 Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 12/29
TectoMT deep transfer Jak pošlu přílohu mailem? poslat Jak adv #PersPron drop příloha n:4 mail n:7 Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 13/29
TectoMT deep transfer Jak pošlu přílohu mailem? poslat Jak adv #PersPron drop příloha n:4 mail n:7 send Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 14/29
TectoMT deep transfer Jak pošlu přílohu mailem? poslat Jak adv #PersPron drop příloha n:4 mail n:7 send How adv Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 15/29
TectoMT deep transfer Jak pošlu přílohu mailem? poslat Jak adv #PersPron drop příloha n:4 mail n:7 send How adv #PersPron n:subj Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 16/29
TectoMT deep transfer Jak pošlu přílohu mailem? poslat Jak adv #PersPron drop příloha n:4 mail n:7 send How adv #PersPron n:subj attachment n:obj Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 17/29
TectoMT deep transfer Jak pošlu přílohu mailem? poslat Jak adv #PersPron drop příloha n:4 mail n:7 send How adv #PersPron n:subj attachment n:obj e-mail n:by+x Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 18/29
TectoMT deep transfer Jak pošlu přílohu mailem? poslat Jak adv #PersPron drop příloha n:4 mail n:7 send How adv #PersPron n:subj attachment n:obj e-mail n:by+x How do I send an attachment by e-mail? Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 19/29
T-lemma and formeme transfer poslat mail n:7? Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 20/29
T-lemma and formeme transfer poslat e-mail (n) 0.32 online (adv) 0.25 address (n) 0.11 online (adj) 0.05... mail n:7 n:by+x 0.23 n:with+x 0.22 adv 0.13 n:via+x 0.04... Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 21/29
T-lemma and formeme transfer poslat e-mail (n) 0.32 online (adv) 0.25 address (n) 0.11 online (adj) 0.05... mail n:7 e-mail n:by+x n:by+x 0.23 n:with+x 0.22 adv 0.13 n:via+x 0.04... Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 22/29
Translation model interpolation P(e-mail mail)= 2 3 MaxEnt mail (e-mail,context )+ 1 3 Static(e-mail,mail) Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 23/29
Translation model interpolation past: P(e-mail mail)= 2 3 MaxEnt mail (e-mail,context )+ 1 3 Static(e-mail,mail) present: P(e-mail mail)= 2 6 OutMaxEnt mail (e-mail, context)+ 1 6 + 2 6 InMaxEnt mail (e-mail, context) + 1 6 OutStatic(e-mail, mail) InStatic (e-mail, mail) Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 24/29
Translation model interpolation future: P(e-mail mail)=w 1 OutMaxEnt mail (e-mail,context )+w 2 OutStatic(e-mail, mail) +w 3 InMaxEnt mail (e-mail,context ) +w 4 InStatic(e-mail, mail) interpolation weights tuned on development set Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 25/29
Translation model interpolation future: P(e-mail mail)=w 1 OutMaxEnt mail (e-mail,context )+w 2 OutStatic(e-mail, mail) +w 3 InMaxEnt mail (e-mail,context ) +w 4 InStatic(e-mail, mail) interpolation weights tuned on development set not available at the moment: tuning in TectoMT development set Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 26/29
Evaluation 40 38 36 34 out in interpol 37.0 33.7 38.9 32 30.6 31.3 30 28 26 24 28.4 27.1 28.3 23.3 26.5 23.0 24.3 22 20 18 21.5 20.4 18.5 18.5 20.4 21.4 16 EN CS CS EN EN ES ES EN EN NL NL EN Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 27/29
Conclusion TectoMT deep transfer system IT helpdesk domain (1000 parallel sentences) domain adaptation by TM interpolation no tuning of interpolation weights consistently positive results, up to +3 BLEU future work tuning of interpolation weights other domain adaptation methods Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 28/29
Thank you for your attention Rudolf Rosa, Ondřej Dušek, Michal Novák, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz Translation Model Interpolation for Domain Adaptation in TectoMT Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics http://ufal.mff.cuni.cz/rudolf-rosa/ Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 29/29