TectoMT: Machine Translation System

Transkript

1 : Machine Translation System Martin Popel ÚFAL (Institute of Formal and Applied Linguistics) Charles University in Prague FEAST (Forum Entwicklung und Anwendung von Sprach-Technologien) November 16, 2010, Saarbrücken, Germany

2 Outline Treex (framework) vs. (MT system) Demo translation step by step Annotation of translation errors Improvements Hidden Markov Tree Models (HMTM) Combining dictionaries Maximum Entropy dictionary Results Examples of translation

3 Treex vs NLP framework MT system Main author: Zdeněk Žabokrtský lemmatization tagging parsing multi-purpose NLP framework Treex MT system coreference CzEng analysis lemmatization tagging parsing PEDT preprocessing treebank conversions named entity r. SMT preproc. alignment (word,tree) etc.

4 Treex architecture input files document reader In-memory document representation (OOP API) block 1... block n document writer scenario output files

5 Translation scheme transfer over the tectogrammatical layer ANALYSIS TRANSFER SYNTHESIS deep syntax: tectogramatical layer t-layer shallow syntax: analytical layer a-layer morphological layer m-layer source language (English) target language (Czech) w-layer

6 Translation scheme rule based & statistical blocks ANALYSIS TRANSFER SYNTHESIS tectogramatical layer t-layer fill formems grammatemes use fill morphological categories query build t-tree dictionary HMTM impose agreement mark edges to contract add functional words analytical layer a-layer analytical functions generate parser (McDonald's MST) wordforms morphological layer m-layer tagger (Morce) concatenate lemmatization tokenization source language (English) target language (Czech) w-layer segmentation

7 Demo Translation Analysis raw text Machine translation should be easy. m-layer machine translation should be easy. NN NN MD VB JJ. a-layer Atr machine Sb translation Pred should Obj be AuxK. Pnom easy

8 Demo Translation Analysis raw text Machine translation should be easy. m-layer machine translation should be easy. NN NN MD VB JJ. a-layer Atr machine Mark functional words Sb translation Pred should Obj be AuxK. Pnom easy

9 Demo Translation Analysis raw text Machine translation should be easy. m-layer machine translation should be easy. NN NN MD VB JJ. a-layer Mark edges to be contracted Atr machine Sb translation Pred should Obj be AuxK. Pnom easy

10 Demo Translation Analysis raw text Machine translation should be easy. m-layer machine translation should be easy. NN NN MD VB JJ. Build t-tree (backbone) t-layer be translation easy machine

11 Demo Translation Analysis raw text Machine translation should be easy. m-layer machine translation should be easy. NN NN MD VB JJ. Fill formems t-layer be v:fin n:attr machine translation n:subj easy adj:compl

12 Demo Translation Analysis raw text Machine translation should be easy. m-layer machine translation should be easy. NN NN MD VB JJ. t-layer n:attr machine Fill grammatemes translation n:subj number = singular be v:fin tense = simple, modality, conditional easy adj:compl degree of comparison = positive

13 Demo Translation Transfer Build target t-tree by cloning source t-layer machine translation n:attr be n:subj v:fin easy adj:compl target t-layer be v:fin translation n:subj easy adj:compl machine n:attr

14 Demo Translation Transfer Get translation variants for lemmas and formems source t-layer be v:fin translation n:subj easy adj:compl machine n:attr target t-layer počítač stroj strojový n:2 n:attr adj:attr překlad převod n:1 být mít v:fin v:inf snadný jednoduchý adj:compl n:1 adv:

15 Demo Translation Transfer source t-layer machine Select the best combination of lemmas and formems translation n:attr be n:subj v:fin easy adj:compl target t-layer počítač stroj strojový n:2 n:attr adj:attr překlad převod n:1 být mít v:fin v:inf snadný jednoduchý adj:compl n:1 adv:

16 Demo Translation Synthesis Build target a-layer by cloning target t-layer být v:fin překlad n:1 snadný adj:compl strojový adj:attr target a-layer být překlad snadný strojový

17 Demo Translation Synthesis Fill morphological categories target t-layer být v:fin překlad n:1 snadný adj:compl strojový adj:attr target a-layer strojový degree = positive být překlad snadný number = singular degree = positive gender = masc. inanim. case = nominative

18 Demo Translation Synthesis Impose agreement target t-layer být v:fin překlad n:1 snadný adj:compl strojový adj:attr target a-layer být number = singular gender = masc. Inanim. number = singular case = nominative gender = masc. inanim. strojový degree = positive překlad number = singular gender = masc. inanim. case = nominative snadný degree = positive number = singular case = nominative gender = masc. inanim.

19 Demo Translation Synthesis Add functional words target t-layer být v:fin překlad n:1 snadný adj:compl strojový adj:attr target a-layer překlad mít. strojový by být snadný

20 Demo Translation Synthesis target t-layer Reorder clitics být v:fin překlad n:1 snadný adj:compl strojový adj:attr target a-layer překlad mít. strojový by být snadný

21 Demo Translation Synthesis Generate wordforms target t-layer být v:fin překlad n:1 snadný adj:compl strojový adj:attr target a-layer překlad měl. strojový by být snadný

22 Demo Translation Synthesis Concatenate tokens for output target t-layer být v:fin překlad n:1 snadný adj:compl strojový adj:attr target a-layer strojový měl. překlad by být snadný Strojový překlad by měl být snadný.

23 Demo Translation Real Scenario MORPHOLOGY: ResegmentSentences Tokenize NormalizeForms FixTokenization TagMorce FixTags Lemmatize NAMED ENTITIES: StanfordNamedEntities DistinguishPersonalNames A-LAYER: MarkChunks ParseMST SetIsMemberFromDeprel RehangConllToPdtStyle FixNominalGroups FixIsMember FixAtree FixMultiwordPrepAndConj FixDicendiVerbs SetAfunAuxCPCoord SetAfun T-LAYER: MarkEdgesToCollapse MarkEdgesToCollapseNeg BuildTtree SetIsMember MoveAuxFromCoordToMembers FixTlemmas SetCoapFunctors FixEitherOr FixIsMember MarkClauseHeads MarkPassives SetFunctors MarkInfin MarkRelClauseHeads MarkRelClauseCoref MarkDspRoot MarkParentheses SetNodetype SetGrammatemes SetFormeme RehangSharedAttr SetVoice FixImperatives SetIsNameOfPerson SetGenderOfPerson AddCorAct FindTextCoref TRANSFER: CopyTtree TrLFPhrases TrLFJointStatic DeleteSuperfluousTnodes TrFTryRules TrFAddVariants TrFRerank TrLTryRules TrLAddVariants TrLFNumeralsByRules TrLFilterAspect TransformPassiveConstructions PrunePersonalNameVariants RemoveUnpassivizableVariants TrLFCompounds CutVariants RehangToEffParents TrLFTreeViterbi RehangToOrigParents CutVariants FixTransferChoices ReplaceVerbWithAdj DeletePossPronBeforeVlastni TrLFemaleSurnames AddNounGender MarkNewRelClauses AddRelpronBelowRc ChangeCorToPersPron AddPersPronBelowVfin AddVerbAspect FixDateTime FixGrammatemesAfterTransfer FixNegation MoveAdjsBeforeNouns MoveGenitivesRight MoveRelClauseRight MoveDicendiCloserToDsp MovePersPronNextToVerb MoveEnoughBeforeAdj MoveJesteBeforeVerb FixMoney OverridePpWithPhraseTr FindGramCorefForReflPron NeutPersPronGenderFromAntec ValencyRelatedRules SetClauseNumber TurnTextCorefToGramCoref SYNTHESIS TO A-LAYER: CopyTtree DistinguishHomonymous. ReverseNumberNounDep. InitMorphcat FixPossessiveAdjs MarkSubject ImposePronZAgr ImposeRelPronAgr ImposeSubjpredAgr ImposeAttrAgr ImposeComplAgr DropSubjPersProns AddPrepos AddSubconjs AddReflexParticles AddAuxVerbCompoundPassive AddAuxVerbModal AddAuxVerbCompoundFuture AddAuxVerbConditional AddAuxVerbCompoundPast AddClausalExpletivePronouns ResolveVerbs ProjectClauseNumber AddParentheses AddSentFinalPunct AddSubordClausePunct AddCoordPunct AddAppositionPunct ChooseMlemmaForPersPron GenerateWordforms MoveCliticsToWackernagel DeleteSuperfluousPrepos DeleteEmptyNouns VocalizePrepos CapitalizeSentStart CapitalizeNamedEntities. FillTagFromMorphcat SYNTHESIS TO TEXT: ConcatenateTokens ApplySubstitutions DetokenizeUsingRules RemoveRepeatedTokens NormalizePunctuationForWMT

24 Annotation of Translation Errors sample of 250 sentences, 1463 errors in total Type Subtype Seriousness Circumstances Source lemma, formeme, gram., w. order,... gram: gender, person, tense,... serious, minor coordination, named entity, numbers tok, lem, tagger, parser, tecto, trans, x, syn,? ANALYSIS 30% SYNTHESIS 3% TRANSFER 67% errors caused by the assumption of t-tree isomorphism 8% other transfer errors 59%

25 HMTM Motivation source t-layer machine Select the best combination of lemmas and formems translation n:attr be n:subj v:fin easy adj:compl target t-layer počítač stroj strojový n:2 n:attr adj:attr překlad převod n:1 být mít v:fin v:inf snadný jednoduchý adj:compl n:1 adv:

26 HMTM Motivation source t-layer machine target t-layer translation n:attr počítač n:2, počítač n:attr, strojový adj:attr,... Select the best label for each node n:subj překlad n:1, převod n:1 be v:fin easy adj:compl být v:fin, být v:inf, mít v:fin, mít v:inf snadný adj:compl, jednoduchý adj:compl,...

27 HMTM in MT Source tree (Czech) ROOT TRANSFER P(optimal_tree) = P E (strojový machine) P T (machine translation) P E (překlad translation) P T (translation be) P E (snadný easy) P T (easy be) P E (být be) P T (be ROOT) Target tree (English) ROOT P E (být have) = SYNTHESIS být P E (být be) = 0.8 be have ANALYSIS překlad snadný P E (překlad arcade) = 0.7 P E (překlad translation) = 0.6 translation arcade easy simple P T (machine translation) = strojový Source sentence: Strojový překlad by měl být snadný. P E (strojový machine) = 0.4 machine P E (strojový engine) = 0.5 engine Target sentence: Machine translation should be easy. P E (source target) emission probabilities translation model P T (dependent governing) transition probabilities target-language tree model

28 Combining Dictionaries new general interface (for lemmas and formems) $dict->get_translations($input_label, $features) returns a list of translation variants including probabilities OOP style, dictionary constructor can take another dictionary (or more) as a parameter hierachy Four basic types of dictionaries: Static plain Context Derivational Combinaional loaded from a file lemma lemma loaded from a file lemma,features lemma translations derived dynamicaly, input dictionary combination of more input dictionaries

29 Hierarchy of lemma dictionaries Backoff 1. MaxEnt (CzEng) Baseline (CzEng) Human

30 Hierarchy of lemma dictionaries Interpolated 0,8 0,1 MaxEnt (CzEng) Human 0,1 Baseline (CzEng)

31 Hierarchy of lemma dictionaries Interpolated MaxEnt (CzEng) Human Prefixes multi-core více-jádrový více-jádro multi-jádrový multi-jádro Baseline (CzEng)

32 Hierarchy of lemma dictionaries Interpolated MaxEnt (CzEng) Human Prefixes Deadjectival_adverbs deaf hluchý hluše necitlivý necitlivě Baseline (CzEng) water voda vodový vodní Nouns_to_adjectives

33 Hierarchy of lemma dictionaries Interpolated MaxEnt (CzEng) Human Prefixes Deadjectival_adverbs Baseline (CzEng) high-water vodový vysoko-vodový vodní vysoko-vodní Hyphen_compounds Nouns_to_adjectives

34 Hierarchy of lemma dictionaries Numbers Suffixes Interpolated MaxEnt (CzEng) Human Backoff Transliterate Verbs_to_nouns Prefixes Deadjectival_adverbs Baseline (CzEng) Hyphen_compounds Deverbal_adjectives Nouns_to_adjectives

35 Maximum Entropy Dictionary Baseline Dictionary p y x = count x, y count x Maximum likelihood estimates (from the training sections of CzEng 0.9) Pruned by thresholds on p(x y) and p(y x) No context used x = source lemma y = target lemma MaxEnt Dictionary p y x = 1 Z x exp i i f i x, y One MaxEnt model for each source lemma (same training data as for the Baseline Dict.) Interpolated with Baseline Dict. (due to pruning) Context features used (x = source context) - local tree context - local linear context - morphological & syntactic categories -...

36 Maximum Entropy Dictionary agree / v:fin tense=past, voice=active negation=0, sempos=v chop, saw, trim, shorten, lumber, hew, lower, delete, crop abolish, cancel,... he / n:subj union / n:with+x cut / v:inf, has_left_child=0, sempos=v, has_right_child=1, tag=vb, position=right, named_entity=0 overtime / n:obj all / adj:attr TRANSFER ANALYSIS SYNTHESIS He agreed with the unions to cut all overtime. Dohodl se s odbory na zrušení všech přesčasů.

37 WMT results: Results BLEU

38 Results BLEU vs. Ranks (WMT 2010) 75 1: Google 70 2: Bojar Rank (human judgement) : PC Translator 6: Eurotran 5: 3-4: Moses (uedin) R² = 0, BLEU (automatic)

39 Results manual ranking ( others)

40 Results manual ranking (> others)

41 Examples of Translation (2009) A miss by an inch is a miss by a mile. Slečna palec je slečna miliónu. I d rather be a hammer than a nail. Spíše bych byl kladivo než nehet. A bird in the hand is worth two in the bush. Pták v ruce je cenný dvakrát v Bushovi.

42 Example of Translation (2011)

43 Sample of MaxEnt Features input_label=nail output_label=hřebík#n (metal nail) child_formeme_n:in+x= is_member= child_formeme_v:fin= next_node_tlemma=down is_capitalized= position=right tense_g=post voice_g=active prev_node_tlemma=drive parent_capitalized= formeme=n:from+x prev_node_tlemma=hammer child_tlemma_few= child_tlemma_remove= sempos=n.denot next_node_tlemma=and formeme_g=v:until+fin child_tlemma_rusty= tag_g=vbp next_node_tlemma=screw output_label=nehet#n (fingernail or toenail) child_formeme_n:poss= child_tlemma_finger= child_formeme_n:of+x= position=left prev_node_tlemma=black child_tlemma_broken= child_formeme_v:attr= formeme=n:at+x formeme_g=n:attr child_tlemma_long= next_node_tlemma=file child_tlemma_false= prev_node_tlemma=false number=sg formeme=n:obj formeme=n:by+x

44 Cooperation is welcome Exploit English a-layer or t-layer for your project Try your dependency parser instead of MST (extrinsic evaluation) Try your machine learning (CRF, SVM, NN,...) instead of our MaxEnt dictionary

45 Thank you