Similarity Explorer

Poem Similarity

Start by searching for a poem (by archive ID like “H II 1, 389” or by a word in the text). You’ll see the most similar poems ranked by whichever method you choose.

Side-by-side comparison

The pair buttons (e.g., “TF-IDF vs Jaccard”) show two rankings next to each other — useful for seeing which matches are found by different methods. Poems appearing in both columns are especially reliable matches.

Combined (RRF)

Merges the methods’ rankings into one fused score; agreement between methods pushes a poem up, but a strong match from a single method can still appear. The algorithm badges (T, J, Tr, A) on each card show which methods contributed.

Poem comparison overlay

Click “Compare” on any match to see both poems verse by verse. Highlighted words show what’s shared:

• Yellow — same exact word
• Blue — same root word (different form)
• Purple — same meaning across languages

Curved lines between the poems connect matching verses. Thicker lines = stronger matches.

Network graph & similarity map

A web of connections shows how your poem relates to its matches and their matches. The similarity map plots where matches were collected geographically.

Tips

• Use the score slider to hide weak matches
• Check Cross-corpus only (on Translation, Alignment, Semantic, Verse Match and Combined) to see connections between the Estonian and the Finnish-side collections
• Click any poem ID to chain-navigate — breadcrumbs track your path
• Deep link: ?poem=H II 1, 389

Verse Similarity

Search for a verse line and see which other verses across the corpus sound similar. Each match is scored by five methods — Jaccard (same words), TF-IDF (same root words), Translation (same meaning across languages), CharBigram (shared letter patterns), and Sentence Embedding (whole-line meaning) — which are then merged into a combined ranking. (See the Overview tab for what each method does.) A separate experimental panel, Semantic neighbours – native embedding, sits below the results and is described at the end of this tab; it is not part of the combined ranking.

Reading a result

Results are a single ranked list, not columns. Each row shows:

• A strip of small coloured squares — one lit square per method that found this match (hover them to see which methods, and at what rank)
• An N/5 count — how many of the five methods agree on this match
• A ×N badge when the same line occurs more than once — identical verses are folded into a single row rather than repeated down the list, and ×N is how many times that exact line appears across the corpus. If several distinct copies (different poems or places) folded into the row, an attestations expander lists each one so you can jump to a specific occurrence.
• A language badge (ET / FI) and the collection place; cross-corpus matches also carry an ET↔FI badge
• Click the arrow to explore onward from that verse — the verse trail tracks where you have been

Sorting the matches

• Best match (default) — balances how many methods agree with how strong each method’s score is, calibrated so no single method dominates. Genuine matches tend to rise to the top, and look-alikes that only one method likes are held back.
• Combined (RRF) — merges the methods purely by the rank each gives a match; agreement helps most, but a verse found by one method alone can still rank high.
• Agreement — orders by the raw number of methods (out of five) that found the match.
• Top raw score — orders by the single strongest method score. Good for surfacing spelling and dialect twins that only the letter-pattern method catches, but it can also float unrelated look-alikes to the top.

Filtering and display

• All / ET↔FI / ET / FI — show all matches, only cross-corpus ones, or only within one corpus
• Place menu — limit to matches from one collection place
• Hide exact repeats — drops the one match that is word-for-word your search line (your search verse re-attested verbatim elsewhere), leaving only the near, non-identical matches. Repeated copies of any other line are already folded into a single ×N row by default (see “Reading a result”), so this control only concerns the search line itself. It greys out when no match is identical to your search line.
• Highlight related words — words spelled identically to one in the search verse are always marked, with no download: the same word repeated within a language, and the same spelling across Estonian and Finnish shown as a likely shared root. Switch this on to also mark the same dictionary word in a different grammatical form (e.g. metsä / metsän) and, across Estonian and Finnish, words that share a root or an English translation; the first time, it loads a one-time word index. It can legitimately find nothing extra to mark when a verse uses dialectal spellings the dictionary does not cover.
• Translations — show an English translation under each verse

Semantic neighbours – native embedding (experimental)

Below the five-method results, an experimental panel finds lines with a similar meaning in any language directly from the original texts, using multilingual sentence-embedding models. It shows two models together, each labelled: LaBSE (Language-agnostic BERT Sentence Embeddings) and mpnet (paraphrase-multilingual-mpnet-base-v2, a Sentence-BERT model), so you can compare them at a glance. It is not part of the combined ranking. Lines that none of the five methods found are highlighted, and the panel opens by itself when it holds such lines.

Each model has its own Show hidden candidates (research view) control that reveals, on click, the stored candidate lines that model’s quality rule rejected, each labelled with the reason it is hidden. Most rejected lines score high on raw similarity yet come from generic phrasings, so read them with caution.

Verse Pairs

Browse the strongest verse-to-verse connections found across the whole corpus, ranked by combined evidence from shared words, rare words, translations, and letter patterns.

• Sort by any algorithm or use Combined (default)
• Check Cross-corpus only to focus on cross-corpus pairs
• Click a pair to see its scores, a map link, and a button that opens both poems
• Switch to By Place Pair to see which collection places share the most verse material
• Select pairs (checkboxes) to expand them on the map — the map then shows where each verse text was collected across the corpus:

Map Expansion (select a pair)

• Main location (large circle) — the place with the most poems containing that verse text
• Spread lines (dashed) — connect the main location to every other place where the same verse appears
• Co-location (purple) — places where both verse texts appear, shown only when such overlap exists
• Green/blue = same-corpus pairs; gold/pink = cross-corpus pairs

Algorithm badges: J (blue) = Jaccard, T (green) = TF-IDF, Tr (teal) = Translation, C (pink) = CharBigram, S (purple) = Sentence.

By Place Pair view shows identical verses + near-matches = combined total. Sort by combined count, identical verses, near-matches, best RRF score, cross-corpus near-matches, or one of the two size-normalized options (in Cross-corpus only mode the sort menu is shorter):

• Size-normalized (small places) — shared verse pairs ÷ √(distinct verse lines recorded in each place), so a thinly collected place that shares a lot ranks above a large archive that shares more in absolute terms.
• Size-normalized (without small places) — the same score, but places with fewer than 1,000 distinct recorded verse lines are left out. The ratio is unstable where almost nothing was collected: two shared lines between two tiny collections can otherwise outrank hundreds shared between two well-collected parishes. This cut applies only while this sort is chosen.

Counts are a lower bound. Place-pair totals are counted over the whole corpus, attribute each verse to its primary recorded place, and exclude self-pairs, so true corpus-wide totals are higher still. (For scale: 288,635 verses carry no recorded place, and 3,601,619 matching verse pairs were skipped because at least one side had no place.) Cross-corpus (ET↔FI) counts additionally inherit language-labelling caveats for border parishes, so treat them as a lower-bound estimate, not an exact count. Matching also compares normalized text (lowercased, spaces collapsed) and does not fold dialectal spelling variants, so one verse spelled differently across dialects (e.g. peningita vs penningita) is counted as two separate texts rather than one match — a further reason the totals understate true parallelism.

Song types

Which song types resemble each other? Choose a song type as the archives file it (ERAB; SKVR, KR) and this view lists the types whose poems are most alike. Each method gives its own list; the combined ranking, shown first, merges several of them into one.

The combined ranking merges these lists: .

From poems to types. Each method first compares single poems or single lines; this view adds those links up over all the poems filed under the two types. Poems filed under both types are left out of that sum, so two types do not look alike only because the archives file the same poems under both; that overlap is shown as a badge instead. The Measure menu sets how the links are added up: Sum of poem pairs adds the strength of every linked poem pair and scales it by the sizes of the two types; Coverage of poems looks at how well each poem of one type is matched by its closest poem in the other type, in both directions, so that a large type does not come first by its size alone.

How much evidence. A pair of types enters a list only if at least three linked poem pairs support it, or two when the poems are in different languages. Where the two types have too few poems for that ever to happen, the pair is shown with a mark for limited evidence. In lists where very large types would otherwise rise near the top of almost every list, the order takes into account whether the two types are close to each other compared with their other similar types. Each list keeps the 30 closest types in the same language and 30 in the other, and the filters work within those 30.

The lists. TF-IDF, Jaccard and CharBigram compare the words themselves; Translation compares the texts through English translations; Semantic compares the meaning of whole poems through the English glosses of their words; Sentence, LaBSE and mpnet compare the meaning of single lines (mpnet is experimental); Alignment matches verses in their order. The combined poem ranking is the Poem Similarity tab’s combined ranking of four methods. The lists marked “(verses)”, the verse match and the line-match lists compare single lines first. The glossary explains each method.

Lines the types share. Shared lines counts the lines both types contain; lines that recur in many different songs count less. Lines found inside shows when lines of one type turn up inside the other, and in which direction. Alignment and shared-line coverage measures how much of one type’s poems can be aligned with the other type’s poems or, failing that, covered by lines they share. These three lists group lines in a way of their own, not the one the Recurring verses and Cross-corpus clusters tabs show, so their counts differ from those tabs.

The archives’ own evidence. Filed together is the share of poems the archives file under both types, and Archive cross-references are the archivists’ own notes, such as “part of” or “compare”. They come from the archives, not from the texts, so the combined ranking leaves them out unless All evidence is ticked. RunoVerse’s suggested type links types through the types RunoVerse suggests for poems; it carries an outlined badge, because a suggestion is not an archive filing.

Reading a row. The squares show how many independent sources place the type among the closest. The bar shows the type’s value in the chosen list (the number beside it) against the highest value on the list; the Minimum slider hides the types whose value is lower than the number shown beside it. Badges mark the same archive heading, the same ERAB category, and the filing: types filed on exactly the same poems are folded into one row, and a separate badge shows when all of one type’s poems are also filed under the other. An ERAB category is a genre, song category or function that the Estonian runosong database ERAB records for its poems, so only types with Estonian poems have one; it counts for a type only when it covers at least one in five of the type’s poems that ERAB has categorised. Why opens the pair’s rank in every list and the poems the link rests on; Compare opens side by side the strongest example in the list, one poem from each type; in a combined ranking, the example comes from the list in which the type ranks highest. A row whose only example is a poem filed under both types offers “Open poem” instead, which opens that poem in the reader.

How often the closest types are the same song. For a blind check, a random sample of song types was drawn. Two language models, Claude Opus and GPT, read poems of each type beside poems of its closest types in the combined ranking, with the type names hidden. Among the five closest types in the same language, both models judged the pair to be the same song, or parts of one song, in of cases; among the three closest in the other language, in ; and among the five closest under All, in . A type that shares only a motif or a theme with the viewed type, or on which the two models disagreed, is not counted as the same song.

Song types created after the blind check were not in its sample, so these figures do not cover the following types: .

Computed, not checked. These lists are computed from the texts, and no one has read a sample of their pairs to check them. A close type is therefore a lead to follow up, not a finding.

Best match is the default order of the combined ranking when it is offered. It reorders the closest types by weighing the combined score, the number of agreeing sources and the ranks in the separate lists, with weights fitted to the blind check’s judgements on half of the sampled types; on the other half of the sample it put fewer unrelated types among the five closest in the same language than the combined score did.

Map and strongest pairs. The map shows where the poems of a type were collected, with those of one similar type beside them; poems without a place on the map are counted in a note under it. A line joins two places when a poem of one type was recorded at one end, a poem of the other type at the other end, and the method chosen in “Similarity by” links the two poems (for the combined ranking, any of the text lists it merges); click a line for the two places, the number of poem pairs and an example pair to compare. Methods that do not pair poems, such as the archives’ own evidence, draw no lines. Strongest type pairs lists the closest pairs among all types for each language pair; sorting them by poems or by places describes the pairs and does not measure similarity. Where more pairs share the value of the last pair shown, those with the most evidence behind them are shown, and a note says how many pairs share that value.

Corpora. JR poems carry no archive song type, so they do not appear here. The section headings of the Kanteletar (KR) are headings of the edition, not song types, and are not listed; six Kanteletar poems filed under “Jauhaessa” are counted with that type.

The page address keeps the chosen type, list and filters, so Copy link shares exactly this view.

Geography

What this map shows: where the same runosongs turn up in more than one place. Each dot is a place where collectors wrote songs down; each line joins two places that recorded the same or nearly the same verse lines (whole poems in Poem Connections and Combined). A thicker line means more shared material. The data spans four runosong corpora — three Finnish (SKVR, the main archive; KR, published Kalevala-metre poetry; JR, the unpublished-poems archive) and one Estonian (ERAB). A green line stays within the Finnish material or within the Estonian material, so it can join places from different Finnish collections; a gold dashed cross-corpus line always joins an Estonian and a Finnish place. Click a dot for the place itself; click a line for what the two places share.

The five views (the buttons above the map):

• Verse Network — every pair of places, compared verse line by verse line. Most links stay within one corpus (green solid); the rarer cross-corpus links are gold dashed.
• Cross-corpus verse clusters, Poem Connections, Combined — all use the whole corpus, but draw only the connections that cross between the corpora.
• Place Focus — pick one place and see lines to every place it connects to.

Colour key (also shown in the legend below the map). Dots: blue = an Estonian place, orange = a Finnish place (darker shades in the cross-corpus views; an emerald dot marks the place you picked in Place Focus). Lines: Verse Network — green solid = same-corpus, gold dashed = cross-corpus; Cross-corpus verse clusters — purple; Poem Connections — teal; Combined — purple solid = shared verses, teal = shared poems, red dashed = both. In the three cross-corpus views every line joins the two corpora, so there is just one line colour.

Verse Network

Lines join collection places that recorded matching verse lines anywhere in the corpus. The metric dropdown controls what the thickness measures:

• Normalized surprise — rewards small places that share material relative to their size: five identical lines shared between two tiny collections signals a stronger link than five shared between two huge ones. (Technically: identical shared verse lines ÷ √(each place's distinct verse lines).)
• Combined count (default) — identical shared verse lines plus near-match verse pairs. These are two different units (cluster types + near-match instances, like the Verse Pairs tab's Combined), so read it as a strength score, not a clean tally.
• Identical verses only — how many exact verse lines (recurring verses — lines attested in many poems across the tradition) are recorded in both places, across the whole corpus. This is the count the Normalized-surprise view is built from, before the size correction.
• Near-matches only — how many fuzzy (not identical) verse-pair matches the two places share, across the whole corpus. Each verse keeps its 5 closest matches per method (Jaccard, TF-IDF, translation, char-bigram, sentence-embedding) above a threshold (0.80 within a language, 0.30 across languages). The separate Verse Pairs browser lists only the strongest pairs as examples; this count is not limited to them.
• Corpus filter — All connections / Cross-corpus only / Same-corpus only.
• Min connections — applies to the count metrics (Combined / Identical / Near-matches), not to the Normalized-surprise ratio. Under Normalized surprise the map shows every pair it can score (at least one shared identical verse and known collection sizes for both places), still limited by Show top.
• Show top (10–500) — keep only the N strongest lines.

Counts are a lower bound. Near-match counts keep only each verse's 5 closest matches per method above the threshold; about 288,000 verses carry no collection place and contribute nothing; each verse is attributed to its primary (first-listed) collection place; self-pairs are excluded; and dialect or orthographic spellings are not folded together, so the same line spelled two ways is at best a near-match, never identical.

Cross-corpus verse clusters

Cross-corpus only. A line joins an Estonian-corpus and a Finnish-corpus place that share cross-corpus verse clusters — groups of near-identical lines that span the language boundary. Dot size and Min clusters reflect each place's total cross-corpus clusters; line thickness and Min connections reflect the clusters this cross-corpus pair shares. The detail panel may also report identical verse-text types shared across the boundary — a different number from the cluster count.

• Corpus (All / ET / FI), Min clusters, Min connections, Top N sliders
• Show all checkbox — draw every connection instead of the filtered top N

Poem Connections

Cross-corpus only. A line joins places where similar whole poems (not single verses) were collected. The algorithm buttons choose how poems are matched:

• RRF — fuses four poem-level methods (TF-IDF, Jaccard, Translation, Alignment); agreement between methods raises the fused score, and the panel shows how many of the four found the pair (two or more is more robust).
• Translation-Pivot — matches poems through their English translations, so it works across the corpora.
• Semantic — matches poems by overall meaning, using word-embedding similarity on the translations. (Thematic and Semantic are single methods, with no agreement count.)
• Min pairs — hide places sharing fewer than N similar poem pairs; line thickness = number of shared cross-corpus poem pairs.
• Show all checkbox — draw every connection.

Combined

Overlays the shared-verse lines and the poem lines on one map: purple solid = shared verses, teal = shared poems, red dashed = a pair sharing both. Clicking a verse line opens the Cross-corpus verse clusters panel; clicking a poem line opens the Poem Connections panel.

• Algorithm (poem matching, as above), Min verse pairs, Min poem pairs sliders
• Show all checkbox (on by default) — toggle the full network vs the filtered view

Place Focus

Pick a single place and see its strongest connections radiating outward.

• Tradition Strength — how much poetic material this place shares with each partner: every unique verse pair the two places share contributes its best similarity score (from four verse-level methods), and those scores are summed — so both how many pairs are shared and how strong each match is push the score up. Larger, better-collected places still tend to score higher.
• Surprising Connections — the same summed score divided by the geometric mean of the two places’ total collected verse lines, so a small place that shares a lot ranks high.
• Place — type-ahead over 800+ collection places. Corpus — All / Cross-corpus only / Same-corpus only. Show — how many of the strongest partner lines to draw (5–60, default 50).
• The emerald dot is the place you picked. Click Focus on this place in any popup to chain-navigate.
• The (ET) / (FI) tag beside each place is a single representative language assigned to that place; language is actually recorded per verse, so a border parish that recorded both Estonian and Finnish verses still shows just one tag. Cross-corpus links are decided by comparing these per-place tags.
• Deep link: ?mode=geography&submode=placefocus&place=Kuusalu

Cross-corpus song types

In the three cross-corpus views, the link Cross-corpus song types on the Song Bridge page opens the list of song types of the different corpora that are linked by shared verses.

Recurring verses

See which parts of a poem recur across the tradition, and which are found nowhere else in the corpus. Each verse gets a color: dark green = widely shared, yellow = found nowhere else, gray = too short to score.

• Browse Most recurrent or Most Unique poems, or search by poem ID
• Click any verse in the color strip to see its 5 closest matches from other poems
• The recurrence index runs 0–100 and rises with three things: how strong a verse's closest parallel is, how many verses in other poems echo it, and how long the line is (short lines match by chance). 80+ marks a widely shared verse; the lower the score the closer to unique, and a verse found nowhere else scores 0.

By cluster

Switch to By cluster at the top to browse the 200 largest groups of recurring verse lines in the tradition — the building blocks singers reused across poems and places. Each cluster shows its member count, the places it reaches, and its corpus split; open one to see sample variants and jump into the Verse Explorer.

Poem Comparison

Compare two runosong poems side-by-side with multi-layer vocabulary highlighting. Words are color-coded by match type:

• Green — same exact wordform in both poems
• Blue — same lemma (dictionary form), different surface form
• Purple — same English translation (cross-corpus match)
• Light blue — unique to Poem A
• Light orange — unique to Poem B

Features

• Verse connections — curved lines drawn between similar verses, based on 5 precomputed algorithms
• Algorithm dashboard — a score panel with 7 similarity scores (five poem-level algorithms — TF-IDF, Jaccard, Translation, Alignment, Semantic — plus Verse Match and Combined RRF)
• Translations — English translations below each verse
• Recurring verses — Dot indicators showing which verses recur across the corpus
• Mini-map — Click to expand; shows collection locations and distance
• Verse popover — Click any verse to see its closest matches and per-algorithm scores

Keyboard shortcuts

• Ctrl+Shift+S Swap poems
• Ctrl+Shift+C Toggle connections
• Ctrl+Shift+T Toggle translations
• Ctrl+Shift+F Toggle recurrence
• Ctrl+Shift+1/2/3 Toggle highlight layers
• Ctrl+Shift+R Random pair
• Esc Clear comparison or close popover

Deep link: ?mode=compare&a=SKVR+I1+1.&b=SKVR+I1+2.

Cross-corpus clusters

Each cluster is a group of runosong verses the pipeline judged near-identical — usually the same verse recurring in both corpora. Every cluster holds at least one Estonian-corpus verse (ERAB) and one Finnish-corpus verse (SKVR, KR, JR).

Soft membership. One verse can belong to several clusters, so the cluster sizes add up to more than the distinct-verse total.

Form vs low overlap. The badge marks how much visible word form the two sides share, not whether they match in meaning. Low overlap is an upper bound — mostly cognates whose spelling drifted apart, or short verses just below the line; genuine meaning-only parallels are nearer 1–2%.

KR? flag. Marks a verse whose poem has Estonian text filed under the Finnish KR corpus — a known limitation, so those counts may be slightly inflated.

The search box and the Type, Language-mix, Min-size, and Sort controls filter the list; the count in the header reflects your current filters.

Glossary