Verse Similarity Analysis

What this page shows

A statistical analysis of verse-level similarity across the entire Finnic runosong corpus (4.3 million verse occurrences, 289,702 poems). The analysis compares how five different algorithms find similar verses and identifies formulaic patterns.

Algorithm Comparison Dashboard

• Jaccard — exact wordform overlap between two verse lines (|intersection| / |union|), with adaptive minimum shared words and IDF weighting.
• TF-IDF — lemma-level cosine similarity, weighted by term rarity across the corpus.
• Translation — cross-lingual similarity via English translation vectors, enabling Estonian-Finnish matching.
• Char Bigram — character bigram overlap using FAISS approximate nearest neighbor search, capturing orthographic similarity between verses.
• Sentence — sentence embedding similarity using all-MiniLM-L6-v2 on English translations, capturing broad semantic resemblance.
• Each card shows coverage (how many verses have matches), average score, and match type distribution (s = same language, x = cross-lingual, w = within-poem).

Cross-Algorithm Discordance

For verses that appear in multiple algorithms, how much do their neighbor lists overlap? High discordance means the algorithms find fundamentally different similar verses. Low overlap suggests the algorithms capture complementary aspects of similarity.

Cluster Distribution Charts

• Language Distribution of Formulaic Patterns — splits the 200 clusters into Estonian-dominant, Finnish-dominant, and bilingual, based on how the verses in each cluster divide between the two languages (a cluster is "dominant" when at least 90% of its verses are in one language).
• Cluster Size Distribution — a histogram of the 200 clusters by how many verse occurrences each contains, with bars colored by their dominant language composition.
• Geographic Spread vs Frequency — a scatter relating each cluster's size to the number of distinct collection places it reaches; clusters that are both large and widely spread are the most universal formulas.

Formulaic Verse Clusters

• The 200 largest groups of near-identical verses found by RRF neighborhood clustering on the combined similarity graph.
• Size = total verse occurrences in the cluster. Places = distinct collection locations.
• Click a cluster row to see its verse variants, geographic spread, and explore links.
• Use language filters and text search to narrow the table. Click column headers to sort.

Geographic Spread Charts

Scatter plot shows how cluster size relates to geographic distribution. Histogram shows the frequency of clusters by number of distinct places. Widely distributed formulas represent the most universal elements of Finnic oral poetry.

Keyboard Shortcuts