Dynamic Development of Vocabulary Richness of Text. Miroslav Kubát & Radek Čech University of Ostrava Czech Republic

Transkript

1 Dynamic Development of Vocabulary Richness of Text Miroslav Kubát & Radek Čech University of Ostrava Czech Republic

2 Aim To analyze a dynamic development of vocabulary richness from a methodological point of view. Q 1 : Do various text segmentations affect the development? Q 2 : Are there significant differences between texts? Q 3 : Are there significant differences between genres?

3 vocabulary richness Example of different vocabulary richness development in two hypothetical texts chapter text_1 text_2

4 vocabulary richness Development of two texts, segmentation 300 tokens anglické_listy AVERAGE VOCABULARY RICHNESS anglické_listy výlet_do_španěl (non-significiant difference according to u-test, α = 0.05) výlet_do_španěl

5 MATTR Distance (d) measurement MATTR I MATTR I+1 d i = [ MATTR i MATTR i ] 1/2 0.1 i 1 + i = i

6 Vocabulary richness MATTR A text is divided into the overlapped subtexts of the same length. TTR is computed for every subtext. MATTR is defined as a mean of particular values. MATTR L = σ i=1 N L V i L(N L + 1) L arbitrarily chosen length of a window, L < N N text length in tokens V i number of types in an individual window

7 Vocabulary richness MATTR a, b, c, a, a, d, f a, b, c b, c, a c, a, a a, a, d a, d, f MATTR 3 = σ i=1 N L V i L(N L + 1) = ( ) = 0.87 L=3 N=7 V i number of types in an individual window

8 Data 16 Czech long texts. 3 genres (travel books, novels, scientific texts). One author (Karel Čapek).

9 Three ways of text segmentation Chapters Paragraphs Sequences of 300 tokens

10 vocabulary richness Development of two texts, segmentation 300 tokens anglické_listy AVERAGE VOCABULARY RICHNESS Anglické_listy Výlet_do_španěl (non-significiant difference according to u-test, α = 0.05) výlet_do_španěl

11 d Development of two texts, segmentation 300 tokens Wilcoxon-Mann-Whitney Test p-value: výlet do španěl anglické listy

12 average d Average distances (d) in texts chapters paragraphs sequences_300

13 average d Average distances (d) chapters paragraphs sequences_300

14 average d Average distances (d) sequences_300 sequences_500 sequences_1000

15 Wilcoxon-Mann-Whitney Test (α = 0.05) Chapter segmentation anglické listy cesta na sever cesta na sever italské listy obrázky z holandska výlet do španěl italské listy obrázky z holandska výlet do španěl hordubal krakatit obyčejný život hordubal krakatit obyčejný život povětroň první parta tovarna na absolutno povětroň první parta tovarna na absolutno válka s mloky život a dílo skladatele foltýna válka s mloky život a dílo skladatele foltýna 29 significant differences objektivní metoda objektivní metoda pragmatis mus směry v estetice pragmatis mus

16 Wilcoxon-Mann-Whitney Test (α = 0.05) Paragraph segmentation anglické listy cesta na sever cesta na sever italské listy obrázky z holandska italské listy obrázky z holandska výlet do španěl výlet do španěl hordubal krakatit hordubal krakatit obyčejný život obyčejný život povětroň první parta tovarna na absolutno válka s mloky život a dílo skladatele foltýna 45 significant differences objektivní metoda pragmatis mus povětroň první parta tovarna na absolutno válka s mloky život a dílo skladatele foltýna objektivní metoda pragmatis mus směry v estetice

17 Wilcoxon-Mann-Whitney Test (α = 0.05) 300 Tokens segmentation anglické listy cesta na sever italské listy obrázky z holandska výlet do španěl hordubal krakatit obyčejný život povětroň první parta tovarna na absolutno cesta na sever italské listy obrázky z holandska výlet do španěl hordubal krakatit obyčejný život povětroň první parta tovarna na absolutno válka s mloky život a dílo skladatele foltýna objektivní válka s mloky život a dílo skladatele foltýna 28 significant differences objektivní metoda pragmatis mus metoda pragmatis mus směry v estetice

18 average d Average distances (d) in genres chapters paragraphs sequences_300 travel_book novel scientific_text

19 u-test, α = 0.05, u 1.96 means significant difference CHAPTERS travel book novel novel x scientific text PARAGRAPHS travel book novel novel x scientific text There is no significant difference between genres in our corpus. 300_TOKENS travel book novel novel x scientific text

20 Chapters "Natural" units. Relatively long units. Appropriate units for linguistic and literary research. Different lengths.

21 Paragraphs "Natural" units. Appropriate units for linguistic and literary research. Very short units for vocabulary richness measurement. Different lengths.

22 Sequences of text with the same length Same length. Good length for vocabulary richness measurement. Artificial units. Last part of a text must be removed. Problematic linguistic and literary interpretation.

23 Preliminary Conclusion and Discussion The longer the sequences of text, the smaller average distances between the subsequent sequences. Both chapters and paragraphs are not suitable for development analysis due to their different length. Sequences of arbitrary chosen length seems to be the best way; however, the artificial character of these units makes the interpretation of the results problematic. Development of vocabulary richness could be used for text analysis. Differences among genres were not corroborated in our corpus. Is there another unit for this analysis?

24 Thank You For Your Attention!

25 References Čech, R. (2015). Development of thematic concentration of text (in Karel Čapek s books of travel). Czech and Slovak Linguistic Review. (accepted) Čech, R., Popescu, I. I., Altmann, G. (2014). Metody kvantitativní analýzy (nejen) básnických textů. Olomouc: Univerzita Palackého v Olomouci. Covington, M. A., McFall J. D. (2010). Cutting the Gordian Knot: The Moving Average Type-Token Ratio (MATTR). Journal of Quantitative Linguistics 17(2), Hřebíček, L. (2000). Variation in Sequences. Prague: Oriental Institute. Köhler, R., Galle, M. (1993). Dynamic Aspects of Text Characteristics. In: Hřebíček, L., Altmann, G. (eds.), Quantitative Text Analysis. Trier: WVT, Kubát, M., Milička, J. (2013). Vocabulary Richness Measure in Genres. In: Journal of Quantitative Linguistics, 20(4): Popescu, I. I. et al. (2009). Word frequency studies. Berlin/New York: Mouton de Gruyter. Tuzzi, A., Popescu, I. I., Altmann, G. (2010). Quantitative Analysis of Italian Texts. Lüdenscheid: RAM.