Statistical Tests of the Voynich Manuscript

Conclusions

In sum, although the Voynich text has properties unlike any natural-language text, most of its statistical properties can be reproduced by a fairly simple combination of writing habits: “build words glyph by glyph following strong habits of glyph continuation, and often write slightly altered versions of words visible in the previous few lines” (Chapters 24–25). This does not settle whether the text has meaning, but it supports both the idea behind the self-citation hypothesis (Timm and Schinner) and a peculiar way of assembling words glyph by glyph.

1. The original hypothesis: does something switch after a run?

At positions where the same word appears twice in a row, we measured how much the glyph distribution, spelling sequences and glyph variability differ between the windows before and after, and compared this with non-run positions on the same page (2,000 permutations).

A small “p (larger change)” would support the hypothesis. All are 0.19 or above, so no condition supports it. Conditions where “p (smaller change)” is below 0.05 (bold) indicate that text before and after runs is more similar. Nine tests were run, so discount each individual p accordingly.

2. Similar words appear close together

Each word was compared with the words 1–30 positions earlier, and counted as “similar” if the spellings differ by at most one glyph (words of 3+ glyphs). The baseline is a random reshuffle of the words within the same page.

Ratio by group (1–10 words back)

Groups with fewer than 400 words are omitted. Every group is at 1.16× or above, so this is not a quirk of a particular scribe or section (smaller groups scatter more).

3. Is there a similar word at the same position in the line above?

The self-citation hypothesis assumes the scribe wrote while looking at the line above. We therefore checked which positions in the line above are most likely to hold a similar word. Line-initial and line-final words were excluded because position itself has strong effects there. Values are ratios to chance.

The horizontal axis is the difference in position within the line (0 = directly above). One line up, directly above is highest (1.28×), but the difference from the neighbouring positions (about 1.20×) is still statistically weak (z ≈ 1.7). What is clear is that similarity fades with distance in lines and is about at chance level five lines up.

4. Comparison with synthetic texts

To see what kind of generating process can reproduce the Voynich statistics, texts of different kinds were measured with the same yardstick, all with the same number of words and the same page division.

• Self-citation model: 95% of words are drawn at random from the manuscript’s words; the remaining 5% are copied from the previous few words. 60% of copies are altered in one or two places (using the one-glyph change patterns actually seen in the manuscript). Tuned to match vocabulary richness and the share of nearby similar words.
• Independent generation (table look-up): imitates only the glyph sequences within words and builds each word independently — a simplified version of building words from tables or grilles.
• Verbose cipher: each Latin letter replaced by one or two glyphs.

● = close to the Voynich (ratios within ±0.04, mean word length within ±8%, others within ±6–30%). The self-citation model and the independent generator use the manuscript’s words and glyphs as raw material, so h2 and the Zipf slope are automatically close. To compare models, look at the top two rows and the share of once-only words.

What this shows:

• The tendency for similar words to sit right next to each other exists neither in English and Latin (which avoid it) nor in independent generation or the shuffle (chance level). It is a defining property of the Voynich.
• The gentle similarity at a distance also exists in English and Latin and is not unique to the Voynich.
• The simple self-citation model matches the rate of identical neighbours but overshoots adjacent similarity, and cannot reproduce the distant tail (0.99×) or the many new words (49% vs 68%).
• The next glyph in the Voynich is highly predictable (h2 = 2.24 bits; English 3.32, Latin 3.30). A glyph-level verbose cipher gets as close as 2.52, but its words become far too long (9.4 glyphs vs 4.4 in the manuscript).

5. Testing the filler (null) hypothesis

Ciphers of the period sometimes mixed meaningless symbols or words (nulls) into the text to hinder decipherment. If repetitions like qokedy qokedy were filler, removing them should leave text that behaves like an ordinary language: neighbours should avoid similar words, and h2 should rise to around 3.3 bits.

We split the pages into two halves, used one half to select the word types that tend to sit next to similar words, and removed all of those words from the other half before measuring. Selecting and measuring on separate data avoids results that look good only because they were tuned on the same data. The split was done both ways and averaged. The comparison is removing the same number of random word types with matched frequencies (mean of 20 runs).

Words most often selected as candidates: chol, qokeedy, qokedy, shol, qokeey, okaiin, shor, qokey. “Adjacent” is the ratio to chance of having a similar word 1–2 positions back; h2 is next-glyph unpredictability (bits).

• Removing the candidates lowered adjacent similarity from 1.24× to 1.07×. This is clearly lower than random removal (1.36×), so the selected words do carry part of the effect.
• Even so, removing about 27% of the text does not push it below 1.0, far from the roughly 0.46× of English and Latin (which avoid similar neighbours).
• h2 rises only from 2.23 to 2.34, about the same as with random removal, and nowhere near the 3.3 of English and Latin.

In conclusion, the simple combination “ordinary text plus specific filler words” is not supported. Both neighbour similarity and glyph predictability are spread across the whole vocabulary rather than confined to a few words. If the filler were at the glyph level (meaningless glyphs added inside words) rather than the word level, this test would not detect it; that case is examined in Chapter 9.

6. Comparison with other languages (93 languages)

We checked with the same yardstick whether any natural language has the tendency for similar words to sit next to each other. The material is a parallel Bible corpus (the Gospel of Matthew onwards, cut to the Voynich’s 34,859 words and page division). It includes languages that use word repetition grammatically (Indonesian, Tagalog, etc.) and scripts that do not write vowels (Hebrew, Arabic, Syriac).

Languages above 1×

z is the deviation from chance (200 within-page shuffles). “Hyphen-joined” recounts hyphenated words such as anak-anak (children) as one word.

• Indonesian is high because reduplicated forms like anak-anak were counted as two words. Counted as one word it drops to 0.39×: the property disappears depending on spelling convention.
• Hebrew (without vowel points) is at 1.25×, almost as high as the Voynich. But it consists only of one-letter-different words (e.g. “said” and “amen”, which differ by one letter, or “hear” with a one-letter prefix), while identical repeats are avoided. Amharic and Zarma show the same pattern.

In the Voynich, both identical neighbours (1.34×, z ≈ 7) and one-glyph-different neighbours (1.21×, z ≈ 8.5) exceed chance, in both Language A and B. Moreover, the raw rate of one-glyph-different neighbours is 3.7%, about twice that of Haitian Creole (1.8%), the highest among the 93 languages.

In conclusion, “neighbours are similar” on its own occurs in languages written without vowels, among others, so this point alone does not show that the text is not language. But the combination of identical repeats and one-glyph variants both gathering next to each other was found in none of the 93 languages. At the least, it is hard to reconcile with ordinary prose copied as it is.

Limitations: the material is a single genre (Bible translations), mostly modern. Repetitive genres are examined in Chapter 7. Ten languages written in Indic scripts were excluded because our word segmentation could not handle them.

7. Can recipe collections and herbals explain it?

Judging from the illustrations, the Voynich is organised like a medieval medical book: herbs, pharmaceutical jars, bathing, astrology. Recipe collections repeat more than ordinary prose, so “neighbours are similar” might be a matter of genre. We therefore measured Latin medical, herbal and culinary texts and a highly repetitive prayer text (the Psalms) with the same yardstick. Other books by the same authors (general prose) are listed alongside so that author habits and genre can be separated.

All values are ratios to chance (adjacent = 1–2 words back, broken down into “identical” and “one-glyph different”). Each text up to 34,859 words. Latin spelling normalised j→i, v→u. z is the deviation from chance (200 within-page shuffles).

• Recipe collections, herbal-virtue texts and the cookbook were all at or below 1× for adjacent similarity. Celsus’s recipe books (0.86×) do not differ from his expository books (1.00×), and Pliny’s herbal books (0.47×) are even lower than his geography books (0.58×).
• Even the repetitive Psalms are at 0.81×; one-letter-different neighbours are limited to set phrases such as deus/meus (God/my).
• No text exceeds 1× for both identical and one-glyph-different neighbours.

In conclusion, matching the genre to the illustrations did not reproduce the Voynich property that identical and one-glyph-different words both gather next to each other. The texts used here are ancient to early-medieval Latin; vernacular recipe collections closer in time to the Voynich were checked in Chapter 8 with a Middle English recipe collection and an Italian cookbook (both 14th century).

8. Comparison with contemporary texts

Texts from around the date of the vellum (1404–1438) were collected regardless of genre and measured with the same yardstick. The last two rows of the table are 14th-century vernacular recipe collections, close to the Voynich in content. Besides Latin, the set includes vernacular texts of the period (Italian, French, Russian, Middle High German). The Nibelungenlied is transcribed in its manuscript spelling (not normalised).

Ratios to chance. Accents removed; Latin normalised j→i, v→u. Each text up to 34,859 words. z is the deviation from chance (200 within-page shuffles).

• Thomas à Kempis’s Imitatio Christi, written in the 1420s like the Voynich, is at 0.91×. It is a devotional manual with many prayers, yet neighbour similarity is at or slightly below chance.
• The highest is Pico della Mirandola at 1.13×, due solely to identical repeats; the text is short (about 10,000 words) and within chance (z ≈ +1.0).
• The 14th-century Middle English recipe collection (Henslow, Medical Works of the Fourteenth Century; endless recipes of the form “take …, grind it, mix with wine and drink”), close to the Voynich in illustrations and date, is at 0.35× and clearly avoids similar neighbours (z ≈ −22.7). The 14th-century Italian cookbook (“take …, wash it well, put it in a boiling pot”) is at 0.61× (z ≈ −4.7).
• All others are below 1×, and no text exceeds 1× for both identical and one-glyph-different neighbours.
• We additionally measured a printed Latin herbal, Aggregator practicus de simplicibus (also known as the Herbarius moguntinus; first edition Mainz 1484, with a woodcut and virtues for each plant). It matches the genre of the Voynich herbal section, is only about 60 years later, and like manuscripts of the period is full of abbreviations. The text is machine OCR of the U.S. National Library of Medicine copy with very many reading errors, so the values are indicative only: adjacent 0.68× (identical 0.44×, one-glyph 0.88×) — it avoids similar neighbours like the other texts.

In conclusion, matching the period did not turn up any text with the Voynich property. Taken together with the earlier comparisons (93 languages, recipe collections and herbals, contemporary texts), the Voynich text is hard to regard as ordinary text in a major European language of the time copied as it is.

9. What if glyphs in fixed positions are decoration?

Voynich words tend to begin with q or o and end with y or n (-aiin etc.), and two or three e or i in a row are common. If such fixed-position glyphs were meaningless decoration, removing them should leave something more language-like: neighbours should avoid similar words (natural-language median 0.53×) and next-glyph unpredictability h2 should rise to about 3.1–3.3 bits.

We removed decoration candidates one at a time or in combination and measured with the same yardstick, comparing with removing the same number of glyphs from random positions (mean of 10 runs).

Ratios to chance and h2 (bits). In parentheses: the same number of glyphs removed at random positions. Reference: the 93-language median is adjacent 0.53, identical 0.39, one-glyph 0.80.

• Whichever candidate was removed, adjacent similarity stayed at 1.17–1.30× and barely moved. Both identical and one-glyph neighbours remained above 1× in every case.
• Even the strongest treatment, removing 44% of the glyphs, left adjacent similarity at 1.17× and h2 at 2.27 bits — not within the natural-language range.
• h2 actually rose more when glyphs were removed at random (up to 2.92 bits): removing the candidate glyphs leaves glyph predictability almost unchanged.

In conclusion, the Voynich’s neighbour similarity and glyph predictability do not reside in peripheral glyphs like initial or final decorations; they permeate the whole word, including its core. An explanation of the form “ordinary text with decorative glyphs added” does not hold either.

10. A page-wide self-citation model

We tried to reproduce the properties that the simple model of Chapter 4 (copy and alter one of the last few words) could not, by widening how the source word is chosen. Each page is written word by word: either a new word is chosen, or an already-written word is chosen as the “source” and written with 0–2 alterations, using the one-glyph change patterns actually seen in the manuscript. The ways of choosing the source compared were:

• From preceding word: from the last few words (close to the Chapter 4 model)
• From same position, line above: from the word at the same or an adjacent position in the line above
• Mixed: a mixture of the preceding words, the line above and anywhere on the page
• Carry-over from previous page: Mixed, plus new words are sometimes chosen from the words written on the previous 1–4 pages rather than from the whole manuscript (close to the form envisaged by Timm and Schinner)

For a fair comparison, the metrics used for fitting were kept separate from those used for checking. The first three types were tuned only on vocabulary richness, the share of once-only words and adjacent similarity. The carry-over type was tuned on all metrics marked “Fitted” in the table (random search over 220 settings). Checking used the mean of three generations with the same settings for every type.

Ratios to chance, or shares/bits. ● = close to the Voynich (ratios within ±0.06, others within ±10%). “Page families” is the rate at which two words on the same page are at most one glyph apart, divided by the rate for two words drawn from the whole manuscript (English 1.16, Latin 1.21).

Findings

• The correspondence with the line above can be reproduced if sources are also chosen from the line above. The carry-over type gives 1.31× directly above and 1.24× adjacent, matching the manuscript’s 1.28× and 1.20×. The type that chooses only from the line above exceeds 3× — far too strong.
• The adjacent combination “identical and one-glyph variants both gather” can be reproduced with a moderate amount of copying: 1.28×/1.28× in the carry-over type (manuscript 1.34×/1.21×).
• Page-level word families are only half reproduced. The manuscript is at 1.60×, the carry-over type only 1.21×, the others 1.0–1.06×. The manuscript shows a gradient of words becoming slightly more similar the closer they are — whole text → same section → adjacent page → same page (one-glyph-difference rate 2.27% → 3.11% for other pages in the same section → 3.51% for adjacent pages → 3.65% on the same page).
• No type satisfies both the large share of once-only words and glyph predictability. The manuscript has as many as 68% once-only words, yet its glyph sequences are highly regular (h2 = 2.24 bits). Every type stayed at 45–58% once-only words; and altering more words disturbs the glyph sequences and raises h2 (the carry-over type fitted to all metrics: 45%, 2.60 bits). The raw rate of one-glyph-different neighbours (manuscript 3.7%) also stayed at 2–2.6% for every type.

Related to the last point, we also measured vocabulary density. In the manuscript, 89% of word types have at least one one-glyph-different relative, with 9.0 on average (English: 49%, 1.5 on average). The models’ vocabularies were about as dense, so the difference lies not in how word types are made but in the rules of alteration.

In conclusion, the skeleton of the self-citation hypothesis (look at already-written words and write slightly altered versions; the range looked at being the preceding words, the line above and the previous page) explains the similarity of neighbours and of lines above and below well. But if the scribe altered words, he did so not by changing one glyph at random but within a strict “word grammar” — rules such as qokedy → qokeedy → qokeey, swapping only specific glyphs in specific places. Identifying and incorporating those rules is the next stage for this model.

11. Alterations that follow a word grammar

The Chapter 10 model altered words by swapping one glyph at random. This disturbed the glyph sequences and made it impossible to combine many once-only words with predictable glyph sequences. We therefore added a “word grammar” to the alterations and repeated the same procedure (fit on the same nine metrics, check on the rest).

• Glyph-grammar type: learns from the manuscript’s words which glyph tends to follow each pair of glyphs (including word start and end), and accepts alterations more readily the better they fit these rules. New forms not in the manuscript can still be made.
• Real-word type: accepts an alteration only if the result is a word that actually occurs in the manuscript. This reuses the manuscript’s vocabulary directly, a somewhat favourable condition.
• Base-form copy type: the glyph-grammar type plus a preference for frequent words as sources — assuming rare forms are throwaway and sources are mainly basic forms.

Best setting from each random search (220–300 settings), mean of three generations. ● = close to the Voynich (ratios within ±0.06, others within ±10%). “9-metric loss” is the sum of squared relative errors over the nine fitted metrics; smaller is closer to the manuscript.

• Following the grammar brought glyph predictability close to the manuscript (glyph-grammar type h2 2.37, real-word type 2.24; manuscript 2.24). The overall loss was also smallest for the glyph-grammar type (0.11 vs 0.16 for the random-edit type of Chapter 10).
• Even so, the share of once-only words was 41–49% for every type, short of the manuscript’s 68%. Favouring frequent words as sources did not change this.

Rare words are not made from nearby words

To identify this barrier, we examined the manuscript directly. If self-citation created rare words, rarer words should more often have a one-glyph-different “source” shortly before them. For each frequency class, we compared the share of words with a one-glyph-different word within the previous 10 words against within-page shuffles.

Ratio to chance of “having a one-glyph-different word within the previous 10 words”; actual shares in parentheses. Words of 3+ glyphs only.

• Of the manuscript’s once-only words, only 4.0% have a one-glyph-different word within the previous 10 words (3.1% by chance) — less than one percentage point above chance.
• In self-citation models, many rare words are born as variants of nearby words, so this ratio rises to 1.84–2.52×. The manuscript is at 1.30×, about the same as English (1.45×) and Latin (1.42×).
• Widening the range to everything earlier on the same page, the share of once-only words with a one-glyph relative is 19.0% in the manuscript, no different from the shuffle (18.9%). Eighty percent of rare words are not one glyph away from any earlier word on their page and cannot be variants of nearby words. Across the whole vocabulary, most rare words do have one-glyph relatives — but those relatives are on other pages.

In conclusion, it is natural to think of the text in two layers. One is a weak local layer of slight similarity to the immediately adjacent words and the line above, well explained by self-citation (writing slightly altered versions of visible words). The other is a context-independent word-building layer that produces most of the vocabulary’s diversity; it seems to create new forms according to a strict word grammar, independently of location, rather than copying nearby words. The strong self-citation hypothesis — “the scribe produced the whole text by continually remixing the preceding words” — is not supported, at least in this form.

12. What the word-building layer is: combining slots

Chapter 11 showed that rare words are made independently of context, not as variants of nearby words. To study the rules of this word-building layer, words were mechanically split into three slots. For words of 4+ glyphs, the first two glyphs are the head, the last two the tail, and the rest the body (ch, sh etc. count as one glyph in the manuscript; the same rule is applied to English and Latin). Common heads in the manuscript are qo, che, ok, ot, da; common tails dy, in, ey, ol, ar.

Heads and tails are chosen almost independently

Strength of association between two slots (normalised mutual information, corrected for chance; 0 = unrelated).

Adjacent slots (head–body, body–tail) are about as strongly associated in the manuscript as in English and Latin. But the association between the separated head and tail is 0.09 — less than a third of English (0.31) and less than half of Latin (0.23). In natural languages a word’s beginning and end are fixed together for each word; in the manuscript the beginning and end appear to be chosen almost separately and combined.

Slot preferences change from page to page

How much each page’s slot distribution differs from the overall (or same section/language) distribution, as a ratio to the case where words are shuffled across pages. 1 = chance.

In the manuscript, which heads and tails are preferred varies strongly from page to page. The bias is especially strong for tails (1.89× relative to the whole text, and 1.37× even within the same section). English and Latin are around 1.1×. The bias for whole words is small (1.09×): page character shows up as preferences for parts — heads and tails — rather than for particular words.

How far does combining slots alone go?

These two properties suggest a simple mechanism: “build words by combining a head, body and tail according to the page’s preferences” (linking only adjacent slots, not head and tail directly). For fairness, a page’s preferences were estimated not from the page itself but from neighbouring pages of the same section. A version adding the self-citation layer of Chapter 11 on top (writing grammatically altered versions of already-written words) was fitted on the same nine metrics and compared (random search over 170 settings, mean of three generations).

● = close to the Voynich (ratios within ±0.06, others within ±10%). “Slot combination only” was not fitted. Unmarked rows are check metrics for all types.

• The many once-only words arise naturally from slot combination. Self-citation models reached at most 58%; combining slots by page preference alone gives 71% (manuscript 68%).
• Page-level word families are also largely explained. Self-citation models reached at most 1.21×; slot combination alone gives 1.37×, and adding self-citation 1.52× (manuscript 1.60×; neither used in fitting).
• Adjacent similarity appears when the self-citation layer is added: chance level (0.97×) with slot combination alone, 1.24× with the layer added (manuscript 1.24×).
• Some points are still off. The correspondence with the line above is too strong (directly above 1.81×; manuscript 1.28×), and glyph predictability is still slightly high (h2 2.42; manuscript 2.24). The 9-metric loss is 0.035, about a third of the previous best (0.11 in Chapter 11).

In conclusion, much of the manuscript text can be reproduced in two stages: (1) build new words by combining heads, bodies and tails according to the page’s preferences; (2) occasionally write slightly altered versions of words already written. Stage 1 produces vocabulary diversity and page character; stage 2 produces neighbour similarity. This does not prove the text is meaningless: a symbol system in which head and tail carry separate information (say, the head a category and the tail a number or case) could produce similar statistics. But ordinary words do not have heads and tails this independent, so if there is meaning, it is more like a code assembled from parts than ordinary words.

13. Fixing the relation to the line above

The two-stage model of Chapter 12 made the correspondence with the line above too strong (directly above 1.81×; manuscript 1.28×). We added knobs for how strongly the word directly above is preferred and how far along the line above sources are taken (directly above only up to 4 words offset), included the correspondence with the line above (directly above and adjacent) in the fitting, and repeated the search (200 settings). Page families and the rate of one-glyph neighbours were kept as check metrics.

● = close to the Voynich (ratios within ±0.06, others within ±10%). Mean of three generations.

The correspondence with the line above came close to the manuscript (now slightly weak). In exchange, page families dropped from 1.52× to 1.35×, and the 9-metric loss roughly doubled from 0.035 to 0.072. We also tried 230 settings of a mechanism borrowing only heads or tails (not whole words) from recent words; none improved anything.

Trade-off

Across all 374 settings explored, 23 had a correspondence with the line above within ±6% of the manuscript, but all of them had a 9-metric loss of 0.105 or more — over three times worse than the overall best (0.028). Mechanisms that “copy” words or parts cannot satisfy the correspondence with the line above together with the other properties.

The manuscript’s correspondence with the line above is probably not a copying trace

We therefore re-examined the manuscript. If each position in a line favours certain word types (say, the second word is often of a certain type), words at the same position in successive lines will be similar without any copying. We re-measured with the baseline changed to “shuffle among words at the same position within the line”.

Ratio to chance of being at most one glyph different from a word 1 line up (or 3–5 lines up). Line-initial and line-final words excluded.

Subtracting position preferences, the special status of directly above drops from 1.28× to 1.22×, hardly different from adjacent (1.19×) or 2–4 words offset (1.15×). Three to five lines up it is 1.01×, about chance. So the manuscript’s “correspondence with the line above” is not the trace of copying the word directly above, but a broad, faint property: nearby lines are similar overall, regardless of position.

In conclusion, the remaining mismatch is attributable to the model creating closeness by copying specific words. The data fit better if the word preferences themselves drift slowly over a few lines in the manuscript, making nearby lines similar as a result. How to produce preference drift at the scale of a few lines — finer than the page-level preferences of Chapter 12 — is the next task. Simply borrowing parts from recent words was not enough; the preference distribution itself must change slightly from line to line.

14. Shuffled page order and the folded sheet (bifolium)

The manuscript is built from folded sheets (each giving four pages) nested into quires, which were then bound. The current foliation was added later, and the order of sheets is thought to have been changed when the book was rebound. But even if the order changed, the two leaves on one folded sheet stay together: in the first quire, for example, f1 and f8 are the two leaves of the same sheet.

Of the results so far, those measured within a page (adjacent similarity, correspondence with the line above, slot independence, etc.) are unaffected by page order. Only the parts using “the next page” or “the previous page” are affected. Using the quire, leaf and sheet positions recorded in the transcription file, we grouped page pairs by their physical relation and compared word similarity, using only pairs with the same section, language and scribe (205 pages).

“One-glyph-diff. rate” is the share of word pairs (one word from each page) at most one glyph apart (words of 3+ glyphs). “Tail-preference difference” is the difference between the two pages’ tail distributions (JSD; smaller = more similar).

Since being in the same quire matters a lot (tail-preference difference about 0.155 between different quires, 0.109 within a quire), we repeated the comparison within quires only. For each page, we ranked how similar the partner page is among the other pages of the same quire, from 0 (most similar) to 1 (least similar). Chance gives a mean of 0.5.

Each pair counted once, using the mean of the ranks in both directions. p is the probability of a mean this low or lower if ranks were uniform (20,000 simulations).

• The opposite leaf of the same sheet is as similar as the recto and verso of one leaf. Although opposite leaves are far apart in the current numbering (f1 and f8, etc.), their rank is at exactly the same level (0.37/0.34).
• Pages adjacent in the current numbering are hardly similar if they are on different sheets (rank 0.45/0.43, close to chance).

In conclusion, the unit at which word preferences cohere is not the current page order but one folded sheet. This fits well with a scribe writing one (opened-out) sheet at a time, making words with the same preferences meanwhile. The manuscript scholar Lisa Fagin Davis also writes that in the herbal section the scribes’ work is “organized by bifolium, not by quire or sequential folios” (e.g. in quire D the outer sheet is by Scribe 1, the next by Scribe 2, and the remaining two by Scribe 1; Manuscript Road Trip, “Voynich Codicology”). The unit of word preference coincides with the unit of scribal work.

This also changes how to read the “neighbouring pages are more similar” effect of Chapter 12: much of it probably came from being on the same sheet or in the same quire rather than from adjacency in the current numbering. It also suggests that the original order might be estimated by chaining similar sheets using their word preferences.

15. Can the original order of the sheets be estimated?

Chapter 14 showed that word preferences cohere at the level of one folded sheet. We therefore applied approach B of the original script (estimate the order by chaining similar items) at the level of sheets rather than pages. For the 50 sheets with 60+ words, the distance between sheets is the difference in head and tail preferences (JSD), compared among sheets of the same language (A or B).

The current order is not random

Rank of the partner among the sheets of the same language group, seen from a given sheet (0 = closest, 1 = farthest; 0.5 by chance), averaged over both directions. p is the probability of a difference this large or larger when sheet numbers are shuffled within quires (5,000 times).

• Sheets of the same quire are close: rank 0.26 within a quire vs 0.53 between quires. Quires preserve coherent content.
• Adjacent nested sheets within a quire are even closer. Sheets whose numbers differ by one (touching each other when nested) are at 0.21, sheets two or more apart at 0.31 (p = 0.021). The nesting order within quires is at least partly original, or adjacent sheets were written consecutively.

Few misplaced sheets

We looked for sheets that are unlike the other sheets of their own quire but like those of another quire. Only one exceeded the 95th percentile of random quire assignments (rank 0.78): quire G, sheet 1 (f49/f56). Its closest sheet is quire D, sheet 3 (f27/f30). In addition, quire A sheet 3 (f3/f6) and quire C sheet 1 (f17/f24) are each other’s closest sheets — a notable cross-quire pair, but not beyond chance.

Estimating the order itself

We searched for the arrangement that chains close sheets with the smallest total path length (the travelling-salesman idea).

“Current-order length” is the total length when following the current quire/sheet numbering. p is the share of 2,000 random orders that are at least as short as the current order. Correlation is the rank correlation between estimated and current order (direction is arbitrary, so the sign is meaningless).

• The current order is clearly shorter than random arrangements. Even for the 24 Herbal A sheets, only 1.9% of random orders are shorter than the current one. Despite rebinding, much of the original arrangement survives.
• But the fine order cannot be determined. The estimated and current orders of Herbal A correlate at only +0.15 (p = 0.46) and hardly agree. The distance differences are small, and a few hundred words per sheet are not enough to fix the fine order of herbal sheets from word preferences.
• That the estimated orders for all Language A and B sheets correlate with the current order (+0.50, −0.52) is mostly due to sections (herbal, biological, recipes, etc.) clustering together.

In conclusion, word preferences tell us which sheets belong together and that the current order broadly preserves the original arrangement, but not with enough precision to rebuild the original order sheet by sheet. The strong candidate for a misplaced sheet is f49/f56; the next subsection compares this with physical clues pointed out by other researchers.

Cross-check with other evidence

We compared the word-preference results with clues from codicological studies (same ranks as above, 0 = closest).

• Sunflower-like plant drawings (agreement). Nick Pelling notes that similar plants are drawn on f33v, f40v and f50r of the Herbal B section and suggests these sheets were once close together (Cipher Mysteries, 2013). By word preference too, the f50 sheet (quire G, sheet 2) and the f33/f40 sheet (quire E, sheet 1) are each other’s closest sheets (rank 0.00). But Herbal B has only seven sheets, so the probability of this happening by chance is about 9%: weak support.
• Tears in the same skin (inconclusive). Pelling writes that sheet pairs with tears in the same orientation (f9/f16 and f10/f15; f35/f38 and f36/f37) may come from the same skin. Their word-preference ranks are 0.19 and 0.29 — ordinary for sheets of the same quire (mean 0.26).
• The f28/f29 sheet (no support). Pelling suggests this sheet, which has a stitched tear, may be out of sequence. By word preference, its rank against the other sheets of its quire is 0.38, with no sign of displacement.
• The f49/f56 sheet (new candidate). In its current position in quire G, its rank against the other sheets is a distant 0.83, but against quire D it is a close 0.38. It does not seem to have been swapped with its closest sheet f27/f30 (which ranks 0.40 in quire D and 0.73 in quire G). Quires D and G have exactly the same pattern of scribes and languages (from the outside: A, B, A, A). We found no study questioning the position of this sheet.

16. Re-check with other transcriptions

All chapters so far used the ZL transcription (Zandbergen–Landini). Transcribers of the Voynich differ in where they divide words and how they distinguish similar glyphs. We therefore re-measured the main metrics using other transcribers’ readings in the Landini–Stolfi interlinear file (text16e6.evt): Takahashi covers almost all pages, the Friedman group most, and Currier about two-thirds. For ZL we also tried treating uncertain spaces (“,” in the transcription) as non-breaks.

Paragraph text only (pages with 20+ words). “Adjacent” is the ratio to chance of a similar word (up to one glyph different) 1–2 positions back, also split into identical and one-glyph-different words. “Opposite leaf” is the Chapter 14 rank (tail preference; 0 = most similar, 0.5 by chance). “Head–tail association” is the Chapter 12 value (English 0.306, Latin 0.226). Currier has fewer pages, so ZL on the same pages is also shown.

• The main results are the same with every transcription: identical and one-glyph-different words both tend to be adjacent (1.22–1.36× and 1.18–1.23×), the share of once-only words, glyph predictability (h2), near-independence of heads and tails, and similarity of opposite leaves. Differences between transcriptions are small — much smaller than the differences from natural languages.
• Joining uncertain spaces changes nothing. The number of word types rises (per 10k words, 2,889 → 3,406), but adjacent similarity is almost unchanged, and the head–tail association actually weakens (0.088 → 0.073).

At one point the Takahashi “identical word” ratio came out low (0.94). This turned out to be an error in our loader, which read the illustration annotations in the interlinear file (such as {plant}) as words. The numbers in the table are after the fix.

In conclusion, the main results of this report do not stem from quirks of word division or readings in the ZL transcription; the same properties are confirmed in at least four major transcriptions.

17. Label words differ from text words

In the astronomical, biological and pharmaceutical sections, one or two words are written next to stars and figures in the drawings, plant parts and jars (labels). If labels name the things drawn, they could be a clue to meaning. We examined 849 label words (3+ glyphs) on the 45 pages that have both labels and running text.

Label words are only weakly tied to the text on the same page

9.9% of labels have an identical word in the text of the same page (8.8% when paired with the text of other pages of the same section; p = 0.08), and 40.0% have a word within one glyph (37.4% by chance; p = 0.03). The same page is slightly favoured, but text words also slightly favour their own page (62.0% vs 57.1%), so this is within what page-level word preferences (Chapter 12) explain. No label-specific link of the kind “this label refers to that word in the text” is visible. Label words also appear in the text far less often than text words do among themselves (25.1%): labels draw on a different vocabulary.

Label words have a different shape

Share of the first glyph of words (ch, sh counted as one glyph). Text is paragraph text, split by position in the line. Restricted to the text of pages with labels, 17.1% of words start with q and 21.6% with o, the same as overall.

• Labels start with o and almost never with q. q, common in the text (16.6%), starts only 1.1% of labels, while 52.6% start with o. The difference in head (first two glyphs) distribution is about 12 times the variation seen when drawing the same number of words from the text. Label heads ot and ok and tails al, ar, am, ly stand out.
• Not because there is no preceding word. Line-initial words also lack a preceding word within the line, yet 12.9% of them start with q and only 13.7% with o. The shape of labels comes from their being a kind of word (labels), not from their position.

Word heads change with how the previous word ends

Words starting with q in the text depend strongly on the last glyph of the preceding word. After a word ending in y, the next word starts with q 27.5% of the time, about 3.8 times the 7.2% after other glyphs. Shuffling words within the page reduces this ratio to about 1.11 (maximum 1.18 over 50 runs). So while a word’s head is chosen independently of its tail within the word (Chapter 12), it is firmly tied to how the previous word ends. The generator of Chapter 12 lacks this link.

In conclusion, label words are a different kind of word from the text and look like names in the drawings, but no correspondence with specific text words was found. Forms beginning with q belong to the flow of the text, especially after words ending in y. In natural languages this recalls word forms used only within sentences (inflected forms, forms with attached articles or conjunctions) or sound changes conditioned by the previous sound (like French liaison). A compositional rule of the scribe (“after a word ending in y, start with q”) would give the same shapes, so this alone does not settle whether there is meaning. But systematic variation of word shape by “label vs text” and by “previous word” is something required both of a meaningful symbol system and of rule-governed composition, and is a clue for narrowing things down next.

18. Adding word-to-word linkage to the model

Chapter 17 showed that a word’s head depends on how the previous word ends (after y, q is likely). Line starts and line ends also have their own biases: line-initial words often begin with d, y, p or t, and 15% of line-final words end in m (0.9% mid-line). We added these mechanisms to the Chapter 12 generator (combine heads, bodies and tails by page preference and occasionally write altered copies of nearby words) to see how much they reproduce.

• Linkage: for each new word, choose whether it is short or long and, if long, its head according to the last glyph of the previous word (or “line start”). To keep page preferences, the section-wide preference given the previous glyph is multiplied by the page’s preference.
• Line end: the tail of a line-final word (or the whole word, if short) is chosen from line-final preferences.

Mean of three runs per model. “y→q” is the share of q-initial words after a y-final word relative to that after other glyphs. “Line start” and “line end” measure how far the distributions of the first glyph of line-initial words and the last glyph of line-final words differ from mid-line words (JSD). “Adjacent: 1-glyph” and “Adjacent: same word” are ratios to chance at 1–2 words back. “Page character” is how much more similar words on the same page are than random pairs from the whole manuscript (Chapter 12).

• Linkage reproduces y→q and the line-initial bias. With copying switched off and every word built according to the linkage, y→q is 3.58× and the line-initial bias 0.20, nearly matching the manuscript (3.81×, 0.20). Page character is preserved.
• Line ends only about half. Even with line-final preferences, the line-final bias is 0.051 against the manuscript’s 0.081. Word shapes at line ends may change by stronger mechanisms than re-choosing the tail (shortening words to fit the line, joining words, etc.).
• Linkage alone does not create similar neighbours. Without copying, identical neighbours are at 0.92× and one-glyph neighbours at 1.00× — chance level. Neighbour similarity still requires writing while looking at nearby words.
• The two mechanisms clash. Adding linkage directly to the Chapter 12 model keeps neighbour similarity but y→q only reaches 2.47×, because copied words ignore the link to the previous word. Refitting everything drops copying to 2%, matching y→q (3.47×) but losing one-glyph neighbours (1.05×) and similarity to the line above.

In conclusion, the scribe apparently respected the linkage rule even when writing altered copies of nearby words — for example, by adjusting the head of the source word to how the previous word ends. The next model must build the linkage rule into copying. In other words, the manuscript’s words satisfy at least four rules at once — internal word structure, page preferences, linkage to the previous word and copying from nearby words — and these rules interlock.

19. Keeping the linkage when copying

Chapter 18 showed that copying nearby words clashes with the linkage rule. We therefore changed how copies are chosen: pick several candidate source words nearby (each altered in 0–2 places) and favour those that fit how the previous word ends. Fit is measured as “frequency of that head after the previous glyph ÷ its usual frequency”. Replacing the head of the copied word would also be possible, but it changes the copy by two or more glyphs and destroys neighbour similarity, so it was not used.

Mean of three runs per model. The top four models keep the best Chapter 12 settings and vary only the number of candidates and the strength of selection (line-final preferences used in all). “Full refit” searches all settings, including number of candidates and selection strength, against the nine manuscript metrics plus four linkage metrics. “Line above” is the ratio to chance of similarity with the word directly above.

• The clash can be resolved. Keeping the Chapter 12 settings and simply choosing the source from eight candidates raised y→q from 2.47× to 3.46× (manuscript 3.81×), while keeping one-glyph neighbours (1.19×) and page character (1.57) (similarity with the line above remains too high, as in the Chapter 12 model). Assuming the scribe “chose words that fit how the previous word ends even when copying” explains both properties at once.
• Line starts and ends still fall short: line-initial bias 0.13 (manuscript 0.20), line-final 0.051 (0.081), about half to two-thirds, because copied words do not shift enough towards line-start/end shapes even when selected from candidates.
• Fitting everything at once is still hard. A full refit matched the linkage metrics well (y→q 3.66×, line start 0.18) but dropped copying to 3%, so neighbour similarity overshot and similarity to the line above (1.04; manuscript 1.28) and page character (1.31; 1.60) were lost. As in Chapter 13, a separate mechanism producing the overall similarity of nearby lines is missing. Glyph predictability (h2) also remains too low in every model (2.45–2.64 bits vs 2.24, i.e. less predictable).

In conclusion, the single device of “choosing, even when copying, a word that fits how the previous word ends” resolves the contradiction of Chapter 18. The text can now be largely reproduced by satisfying four rules at once: (1) internal structure (head, body, tail), (2) page preferences, (3) linkage to the previous word, and (4) writing while looking at nearby words. The remaining large differences are the overall similarity of nearby lines (Chapter 13) and glyph predictability; both might be explained by word preferences drifting slowly over a few lines.

20. Word preferences drifting from line to line

Two large differences remained in the Chapter 19 model: the overall similarity of nearby lines (Chapter 13) and glyph predictability (h2). We first measured in the manuscript how similar words are as a function of how many lines apart they are (up to one glyph different; ratio to within-page shuffles).

“1 line” = words one line apart (any column); “directly above” = the word at the same position one line up. Mean of six runs per model. “9-metric loss” is the difference from the manuscript used since Chapter 12 (smaller = closer).

In the manuscript, similarity fades slowly over several lines: 1.18× at one line, 1.06× at two, 1.06× at three. The Chapter 19 model is strong only at one line (1.33×) and about chance from two lines on. This is because the “copy from the line above” mechanism concentrates on the word directly above; removing it alone drops directly-above similarity from 1.91× to 1.11×.

We then added slowly drifting word preferences: on each page, the head and the tail each carry a “current preference weight”, which changes randomly from line to line while partly carrying over the previous line’s weight. New words choose heads and tails with these weights applied. Copying from the line above is not used.

• Similarity 2–3 lines apart came closer to the manuscript. With drift, it is 1.06× at two lines and 1.04× at three, closer than without drift (1.03×, 1.02×). Other metrics hardly worsen (9-metric loss 0.161 vs 0.144 without drift).
• One line apart and directly above are still somewhat short: 1.13× and 1.14× vs the manuscript’s 1.18× and 1.28×. Stronger drift (preliminary comparison, mean of three runs) reaches 1.18× at one line and 1.20× directly above, but identical neighbours become too frequent (1.68×) and the 9-metric loss worsens about 1.7-fold. A fully automatic search over all settings was also tried, but it went in the direction of creating line similarity by more verbatim copying, breaking one-glyph neighbours and page character, so it was not adopted.
• What drifts in the manuscript is a finer preference. Splitting the similarity of nearby lines into identical and one-glyph-different words, both contribute equally in the manuscript (1.18× each at one line). The model’s drift mainly increases identical words (1.22×), with one-glyph variants only at 1.10× (this breakdown from a single generation). The scribe’s preferences seem to drift at the level of one or two glyphs within words, not of whole words.
• Glyph predictability is unchanged by drift. h2 stays at 2.58 (manuscript 2.24). Decomposing the gap, it is spread over the inside of short words (3 glyphs or fewer), the junctions between head, body and tail, and the first glyph of words. Choosing the tail to fit the head in words with an empty body lowers h2 to 2.53 and the 9-metric loss to 0.105, but most of the gap remains.

In conclusion, slowly drifting word preferences partly explain the broad, faint similarity of nearby lines without breaking other properties. But the two differences were not closed at the same time. To explain the similarity of nearby lines fully, preferences at a finer glyph level, not whole heads and tails, must drift. Glyph predictability is a separate problem from line drift: the rules of how glyphs are assembled within words (especially in short words and at part junctions) must be captured more finely than in the current model.

21. Comparison with “scripts built from parts” like Hangul and Chinese

Voynich words are built by combining head, body and tail parts (Chapter 12), with strict rules for how parts combine. This resembles Hangul, where an initial, a medial and a final combine into one syllable block, and Chinese, where a syllable consists of an initial and a final (pinyin). Jorge Stolfi also argued, from the distribution of word lengths, that the Voynich may be a “one word = one syllable” language like Chinese, Vietnamese or Tibetan (Stolfi, “Chinese redux”). We therefore converted the Korean, Chinese and Vietnamese Bibles into sequences of parts (letters) and applied the same yardstick under several segmentations.

34,859 words each. Korean Hangul decomposed into initial, medial and final; Chinese as toneless pinyin (zh, ch, sh, ng counted as one letter); Vietnamese in its standard spelling with spaces between syllables (vowels with tone marks counted as one letter). For Korean, Chinese and Vietnamese the second half of the Bible is used, to avoid the genealogy at the start (repeated personal names). “Same” and “1-glyph” are ratios to chance at 1–2 words back (words of 2+ letters). “Head–tail” is the association between the first two and last two letters of words of 4+ letters (as in Chapter 12; closer to 0 = more independent). “Linkage” is the association between the last letter of a word and the first letter of the next. “Distinct” is distinct words per 10,000.

• Glyph predictability is at the level of a single syllable. Segmenting Korean and Chinese syllable by syllable gives h2 of 2.32 and 2.35, close to the Voynich (2.24); English and Latin are about 3.3. A script that places parts into fixed slots to form syllables makes glyph sequences as predictable as the Voynich.
• But the number of distinct words is at the level of words. Syllables come in few types — only 316–495 per 10,000 (Chinese has only about 1,300 syllables even with tones distinguished) — against 2,889 for the Voynich. Joining syllables in pairs or counting whole words brings the number of types to the Voynich level, but h2 immediately rises to 2.8–3.6. The Voynich was the only text combining “glyph sequences as predictable as single syllables” with “as many distinct forms as words”.
• Head–tail independence resembles “two syllables joined”. The Voynich’s head–tail association is 0.088; the closest was Chinese mechanically joined two syllables at a time (0.100) — as if two independently chosen parts were placed side by side.
• Word-to-word linkage is at the level of syllable boundaries inside words. The Voynich’s linkage to the previous word is 0.070, as strong as between Korean syllables (0.079) and two to four times that between Korean eojeol (0.032) or English words (0.016). Voynich spaces behave less like ordinary word boundaries than like syllable boundaries within words.
• Neighbour similarity appears under no segmentation. No segmentation made both identical and one-glyph-different words likely to be adjacent.

In conclusion, Voynich words have the “parts placed into fixed slots” construction of Hangul syllable blocks and Chinese syllables. Glyph predictability and the strength of word-to-word linkage in particular fit syllable-level writing well. But taking “one Voynich word = one syllable” gives too many word types, and “one word = two syllables” makes glyph sequences too unpredictable. Neither view explains the tendency of similar words to gather next to each other. The Voynich seems to be a system slightly different from any natural writing: around two strictly assembled, syllable-like parts combined independently, with the joins continuing across word boundaries.

22. Reproducing both “identical” and “one-glyph” similarity between nearby lines

Chapter 20 showed that in the manuscript, identical and one-glyph-different words contribute equally to the similarity of nearby lines (both 1.18× at one line apart), while drifting preferences in the model mainly increase identical words. We therefore tried several finer-grained mechanisms.

• Drifting edit fashion: apply to new words one of the one-glyph changes seen between nearby manuscript words (add an e, change o to a, etc.), with the preference for each kind of change drifting slowly from line to line.
• Drifting only one part: apply drift to only one of head, body and tail, leaving the others free.
• Copying from the previous few lines: instead of copying “the word directly above”, which failed in Chapter 13, choose a source word anywhere in the previous few lines and almost always alter it in one or two places.

Mean of six runs per model. Values by line distance are ratios to chance (within-page shuffle) of word pairs at that distance being identical (or one glyph different). “Adjacent” = 1–2 words back. “9-metric loss” is the difference from the manuscript used since Chapter 12.

• Drifting preferences lean towards identical words at every level. Whether the edit fashion or only the tails drift, one-glyph similarity at one line stays at 1.09–1.10×, short of identical words (about 1.22×). New words are so diverse that the “original form” of an altered word is rarely nearby, so one-glyph pairs seldom arise.
• “Copy with alteration from the previous few lines” narrows the gap. Basing copies on any word in the previous three lines and almost always altering them gives 1.22× identical and 1.14× one-glyph at one line apart: one-glyph rises and the gap with identical words narrows (0.12–0.13 with drifting preferences, 0.08 here). But two–three lines apart becomes too strong (1.18×) and four or more lines apart falls below chance. Widening to the previous five lines and adding weak Chapter 20 drift gives 1.26/1.15× at one line (identical still a little strong), 1.16/1.08× at two–three lines and 1.04/1.03× at four–six lines — the closest line-distance profile to the manuscript so far. Adjacent identical (1.31×) and one-glyph (1.24×) words are also almost the same as in the manuscript (1.34×, 1.21×).
• Remaining differences. Similarity with the word directly above is 1.17×, still short of the manuscript (1.28×). Glyph predictability (h2) is unchanged.

In conclusion, the similarity of nearby lines fits “the scribe based words on those visible in the previous few lines and altered them slightly almost every time” better than “the scribe’s preferences drifted slowly at the glyph level”. Chapter 13 failed because the source was limited to the single word directly above; allowing sources anywhere in the previous few lines brings both identical and one-glyph similarity between nearby lines much closer to the manuscript without spoiling other properties. At one line apart identical words are still a little too frequent, so the reproduction is not complete. This supports the idea behind Timm and Schinner’s self-citation hypothesis (copying and slightly altering visible, already-written words), with the range of sources being the previous few lines.

23. Closing the gaps one at a time (five iterations)

Starting from the Chapter 22 model, we repeated “measure in the manuscript → pick the largest discrepancy → investigate the cause and fix it” five times. The table shows where each round ended.

Mean of six runs per model. “1 line, same / 1-glyph” are ratios to chance of words one line apart being identical / one glyph different; “Adjacent” = 1–2 words back. “Line end” is the bias of the last glyph of line-final words (JSD). “Page character” as in Chapter 12. “9-metric loss” is the difference from the manuscript used since Chapter 12.

Round 1: where are the copy sources?

We counted, in the manuscript, where the possible “sources” of similar words (up to one glyph different) are located. With the usual baseline, sources appeared strongly concentrated at roughly the same column (±3 words). But with a baseline shuffling only among words at the same position in the line, the column effect almost vanished (1.1–1.2× at any column one line up): the apparent column effect was due to line-start and line-end word shapes. The real remaining effect is distance in lines: the excess of similar words is concentrated one line back, about a third remains two to three lines back, and almost none from four lines back. We therefore took sources from the previous four lines, favouring nearer lines (halving for each line further back). Though based on the manuscript measurement, this change alone did not reduce the line-distance loss (0.187 → 0.218; it appeared to with three-run averages, but with six runs this was within random fluctuation), and identical adjacent words dropped to 1.23× (manuscript 1.34×).

Round 2: identical adjacent words

In the manuscript, between a word and earlier words on the same line, identical words are enriched more (1.37×) than one-glyph-different words (1.19×): adjacent copies are often verbatim. Adding occasional verbatim copying of an earlier word on the same line (about 0.2% of all words) gave adjacent identical 1.35× and one-glyph 1.15× (manuscript 1.34×, 1.21×). Verbatim copying of just 0.6% of words shoots this up to 1.7×, so this property is determined by a very small amount. The line-distance loss also dropped to 0.100, the smallest of the five rounds.

Round 3: line starts and ends

Round 1 suggested that the remaining shortfall in similarity with the word directly above comes from line-start and line-end word shapes. Investigating, the model had the same share of m-final words at line ends as the manuscript, but three times as many m-final words mid-line, because word preferences were taken from a distribution mixing all line positions. Building mid-line words from preferences estimated on mid-line words only, and favouring position-typical words at line starts and ends, brought the line-final bias to 0.075 (manuscript 0.081) and the 9-metric loss down to 0.112.

Round 4: glyph predictability

Decomposing the h2 gap (2.50 vs 2.24) gave about 0.15 bits at part junctions (head→body, body→tail, head→tail) and about 0.06 bits inside short words; the latter comes from altering copies in one or two places, producing short words with shapes absent from the manuscript. Making four candidates for each new word and favouring those with manuscript-like glyph sequences lowered h2 to 2.30 and the 9-metric loss to 0.054, the best so far. But once-only words fell to 63% (manuscript 68%) and adjacent one-glyph similarity weakened to 1.07×.

Round 5: reconciling predictability and diversity

The dilemma of round 4 is exactly the manuscript property found in Chapter 21: “predictable glyph sequences, yet many distinct words”. Two methods were tried: a) judge candidates only by the glyph sequence at part junctions, not the whole word; b) choose only the head and write the rest glyph by glyph following the manuscript’s glyph-continuation statistics (three-glyph units). a) lowered h2 to 2.30 but once-only words stayed low at 63%; b) restored once-only words to 70% and fitted the line-distance profile well (loss 0.148), but h2 stayed at 2.42, and because the page-specific tail preferences were weakened, page character fell to 1.21.

In conclusion, five iterations largely explained three things as mechanisms of the manuscript: where copy sources are (the previous few lines, any column), occasional verbatim copying of the immediately preceding word, and the shape of line-final words. But similarity with the word directly above stayed at 1.10–1.18× in every round, short of the manuscript’s 1.28×. What remained to the end was satisfying “glyph predictability”, “many distinct words” and “page-specific preferences” simultaneously: with the current part-combination or glyph-continuation statistics, fixing one breaks another. The scribe must have had more precise word-building rules satisfying all three, and identifying them is the next major task.

24. An overnight autonomous improvement loop

To solve the problems left in Chapter 23 (glyph predictability h2, the number of distinct words, page-specific preferences, and the 1.28× similarity with the word directly above), an improvement loop was run autonomously from 21:00 on 16 October until the next morning. The procedure:

• Every evaluation is a six-run mean. Each candidate was generated six times with different random seeds and judged on the mean. The same six seeds were used for all candidates, so candidates are compared fairly.
• Objective J. The sum of the 9-metric loss used since Chapter 12, the loss on similarity by line distance (including directly above), the loss on the four linkage metrics (weight 0.5), and the loss on the four key targets (one unit = h2 ±0.05, once-only words ±2 points, page character ±0.08, directly above ±0.05). Smaller is closer to the manuscript; a key target counts as “reached” when it falls within its range.
• Rule for getting stuck. If there was no improvement twice in a row (in automatic search: two blocks of eight candidates), the largest remaining component of J was identified, the cause analysed, and the approach switched to a different mechanism or different settings.
• Final check on fresh seeds. The top candidates were re-evaluated on six seeds not used in the search (201–206), to make sure the result was not tuned to the particular seeds.

220 candidates were evaluated in total. The settings and results of every candidate are in the attached full history (history.csv), the final model’s settings in final_params.json, and the record of the automatic search (including diagnoses and changes of approach, in Japanese) in search_log.txt. The progress was as follows.

The candidate with the smallest J at each stage. “Reached” lists the key targets within range (h2, once-only words, page character, directly above).

Finding 1: writing glyph by glyph reconciles predictability with diversity

To check the cause of the Chapter 23 dilemma (lowering h2 reduces once-only words), words were generated independently from the manuscript’s glyph-continuation statistics alone. h2 matched the manuscript exactly (2.235), and once-only words were 72% with two-glyph context and 65% with three-glyph context, bracketing the manuscript’s 68%; the number of distinct words was also at the manuscript level. The property described in Chapter 21 as unique to the Voynich arises naturally when words are written glyph by glyph; the failure to combine the two came from combining parts. New words were therefore switched to glyph-by-glyph writing, with the last two glyphs of the previous word included in the context. Word-to-word linkage such as y→q then appears without any special mechanism.

Finding 2: page character is a product of a section effect and a page effect

With glyph-by-glyph writing, page character dropped to about 1.3. Investigating, the manuscript’s 1.60 turned out to be the product of a “same section” effect of 1.42× and a further “same page” effect of 1.14×. The models up to Chapter 23 reproduced only the section effect and had no page effect (1.03×). A mechanism making words with larger families of similar words on the page more likely to be chosen as copy sources (“family attraction”) restored the page effect.

Finding 3: after-the-fact replacements were breaking the linkage

Round 3 brought no improvement and triggered a diagnosis. Stronger selection of copies did not move the linkage metrics and worsened h2, so the cause was probably not the copies; new words alone were re-measured. Glyph-by-glyph writing alone matched the manuscript in linkage, y→q and line-initial bias; what broke them was the mechanism replacing line-initial and line-final words after the fact. Dropping the replacements, writing line-final words from the start with “line-final glyph-continuation statistics”, and not copying at line edges (round 4) brought all four key targets within range for the first time.

Finding 4: retrace the previous word and diverge

Adjacent one-glyph-different words (manuscript 1.21×) were too few, so a mechanism was added that “retraces the first few glyphs of the previous word and continues with glyph-continuation statistics (rewriting if the result is identical)”. Doing this for about 2% of words brought adjacent one-glyph similarity to the manuscript level (round 5). Adjacent pairs like qokedy qokeedy fit this way of writing well.

Finding 5: choose copy sources by glyph-continuation statistics too

The automatic search twice repeated a diagnosis of the same component (word-to-word linkage) without progress, so the diagnostic rule was changed to “move on to the next component if the previous focus failed”, and the cause of the weak linkage was investigated. Copy selection still used the part-based “head frequency ratio”, which favours rare heads. Choosing instead by the probability that the word’s first two glyphs follow the previous word (glyph-by-glyph statistics) (round 6) lowered the 9-metric loss to 0.007. Weakening family attraction to fix the now too strong page effect (round 7) brought all metrics into line.

What did not work

• Strongly selecting copies by the part-based criterion (round 3): h2 worsened.
• Choosing copy sources by a glyph-continuation ratio that does not favour frequent words (R7-4): linkage broke down and J worsened greatly.
• Drifting head preferences, changing temperature, or using part combination for half the words: the page effect did not return.
• The improvements of the automatic search were mostly tuning to the random seeds. Automatic search 3 lowered J from 0.084 to 0.058 on the search seeds, but on fresh seeds it was 0.086, no different from its starting point R7-2 (0.083). Comparing dozens of candidates on fixed seeds selects settings that happen to suit those seeds. The small improvements of searches 1 and 2 are of the same size. All major progress came from new mechanisms derived from analysing causes.

25. The final model and what it shows

How the final model works

The model reached overnight writes each page word by word according to the following rules (numbers are the final settings).

1. New words (about 65% of words not at line edges, and all line-initial and line-final words) are written glyph by glyph. Each glyph is chosen from the previous two or three glyphs (including the last two glyphs of the previous word and the word boundary) according to the manuscript’s glyph-continuation statistics, mixing statistics for the whole section with those of nearby pages (3 pages on each side). Line-final words are written with line-final statistics.
2. Nearby words are copied (about 35% of words not at line edges). Sources are mainly anywhere in the previous 3 lines (nearer lines more often; slightly closer columns favoured) or anywhere on the page, where words with larger families of similar words are favoured. When copying, 12 candidates are picked and altered in one or two places, and the one whose beginning most readily follows the previous word is chosen. No copying at line starts or ends.
3. An earlier word on the same line is occasionally copied verbatim (about 0.5% of words not at line edges).
4. The beginning of the previous word is retraced and the rest rewritten (about 3% of new words).

Main settings of the final model (all values in final_params.json).

Check on fresh random seeds

The search kept using the same six seeds, so it risked selecting settings that happen to suit them. The top candidates were therefore re-evaluated on six seeds not used in the search.

“J (search seeds)” uses seeds 101–106; “J (fresh seeds)” uses seeds 201–206.

The final model (R7-2 from round 7) has J = 0.083 on the fresh seeds, with all four key targets within range (h2 2.22, once-only words 69.5%, page character 1.60, directly above 1.26). The best of automatic search 3 looked far better on the search seeds (J = 0.058), but on fresh seeds it is 0.086, no different from R7-2. Discounting seed tuning, the level reached by the mechanism changes up to round 7 is the night’s result. The starting point (Ch. 23, round 3) has J = 1.064 on fresh seeds, so J fell to about 1/13.

Scorecard: every yardstick so far

The final model was compared with the manuscript on every yardstick used in this report, including metrics not used for fitting (mean of the six fresh seeds).

Metrics without a mark in the “Fitted” column were not part of the search objective and serve as checks. “Origin of rare words” is the Chapter 11 test (ratio to chance of once-only words having a one-glyph-different word within the previous 10 words). “Word-length distribution difference” is the JSD from the manuscript (smaller = closer). The page and section effects are estimated from sampled word pairs and fluctuate by about ±0.04 (the difference from the manuscript values 1.14 and 1.42 in Chapter 24 is within this range).

Most metrics not used for fitting are also close to the manuscript. In particular, the head–tail association (0.076; manuscript 0.088) is closer to the manuscript than in the earlier models that combined parts independently (about 0.03), showing that the “nearly independent head and tail” property of Chapter 12 also arises naturally from glyph-by-glyph writing. The split between page and section effects (1.15, 1.43; manuscript 1.14, 1.38) and similarity one line up, adjacent (1.22; manuscript 1.22) also match.

What this model shows

Most statistical properties of the manuscript text can now be reproduced by a combination of fairly simple writing rules. In summary, the manuscript fits a way of writing like this:

• The scribe assembles words glyph by glyph following strong habits of glyph continuation. These habits differ slightly by section and by nearness of pages, take a different form at line ends, and the ending of the previous word influences the beginning of the next.
• The scribe often writes slightly altered versions of words visible in the previous few lines. Source words that fit the link to the previous word are chosen, and forms used repeatedly on the page are favoured.
• Occasionally the scribe retraces the beginning of the previous word and diverges into another form, or repeats a word as it is.

Three caveats. First, the model is built from the manuscript’s own statistics (glyph continuation, section and nearby-page biases, line-final shapes, etc.), so it shows not that the manuscript was written this way, but that assuming so is statistically consistent. Second, it does not prove the text is meaningless: meaningful content written in a peculiar system assembling syllable-like parts, by a scribe who wrote while looking at nearby words, could give similar statistics — although such a way of writing differs from every natural-language text examined (Chapters 6–8, 21). Third, evaluations fluctuate with the random seeds, and small differences (up to about 0.05 in J; Chapter 31) are meaningless.

Remaining differences. (1) Origin of rare words: in the model, once-only words have a one-glyph-different word within the previous 10 words at 1.87× chance, higher than the manuscript (1.32×). In the model, too many new forms arise as altered copies of nearby words; the scribe may have more often altered copies into forms used elsewhere (the direction of the “real-word type” of Chapter 11). (2) y→q is 3.13×, short of the manuscript’s 3.81× (overall word-to-word linkage is 0.197, the same as the manuscript’s 0.192, so only the specific y-then-q combination is weak). (3) One-glyph similarity one line apart (1.11; manuscript 1.17) and similarity 3–10 words back (1.12; manuscript 1.18) are slightly weak. These are the next tasks.

26. Comparison with 23 contemporary recipe collections

The contemporary vernacular recipe collections compared in Chapter 8 were only two (Middle English and Italian). On GitHub we found a dataset of medieval and early-modern recipe collections, mainly from the German-speaking area, transcribed in their original spelling (neongrau/fyndling-tei; transcriptions based on Monumenta culinaria, the Graz Corpus of Recipes of the Middle Ages (CoReMA), and others). We measured the 23 collections dated before 1560 with at least 3,000 words using the same yardstick. Only the original text of each recipe was used, not the modern German translations included in the dataset.

Recipes of each collection in manuscript order, up to 34,859 words. Values are ratios to chance (words shuffled within the page). z is the deviation from chance (200 shuffles). *The transcription of The Forme of Cury contains an 18th-century editor’s glosses pairing old and modern spellings (e.g. “morter. Mortar.”), which inflates the one-glyph value; indicative only.

• All 23 avoid putting similar words next to each other (adjacent 0.16–0.59×). This includes a Latin recipe collection written in the 1430s, the same decade as the Voynich vellum (0.23×), the Middle High German Daz buoch von guoter spîse of about 1350 (0.38×), and a Czech collection of 1535 (0.37×), close to Prague, where the Voynich was in the 17th century.
• None exceeds 1× for both identical and one-glyph-different neighbours. Identical neighbours are at most 0.50× in every collection.
• Next-glyph unpredictability h2 is 2.91–3.42 bits, clearly higher than the Voynich (2.24).

In conclusion, the explanation “neighbours are similar because recipe texts repeat stock phrases” does not hold even for handwritten recipe collections of the same period and region as the Voynich, in unstandardised spelling. Together with the results up to Chapter 8 (93 languages, Latin recipe collections and herbals, contemporary texts), the neighbour similarity of the Voynich is a property distinct from texts of this kind in Europe at the time.

27. Middle High German medical and herbal texts (ReM)

After the recipe collections of Chapter 26, we measured medical and herbal texts. The source is the Reference Corpus of Middle High German (ReM: Referenzkorpus Mittelhochdeutsch 1050–1350, Version 2.1), which transcribes manuscripts of 1050–1350 in their original spelling, including abbreviation marks and long s (ſ). From it we extracted the medical, herbal and lapidary texts. For comparison we added, from the same corpus, the Physiologus (a book on the natures of animals), the encyclopaedic dialogue Lucidarius, and the cookbook Daz buoch von guoter spîse, which was also measured in Chapter 26.

Values are ratios to chance (words shuffled within each page); z is the deviation from chance (200 shuffles). Values measured on the manuscript spelling (a word with abbreviation marks counted as one spelling) are shown next to values after changing ſ to s and removing abbreviation marks. “Small medical texts” pools ten short texts (a few hundred words each): the Prüll and Innsbruck medical books, herbal, lapidary and recipe texts.

• All the medical and herbal texts also avoid putting similar words next to each other (adjacent 0.44–0.52×). Including the control texts the range is 0.44–0.63×, and normalising the spelling does not change this.
• None has both identical and one-glyph-different words above 1×. h2 is 2.98–3.33 bits, clearly higher than the Voynich (2.24).
• The cookbook Daz buoch von guoter spîse is below 1× both in the transcription used in Chapter 26 (0.38×) and in the ReM transcription (0.59×), confirming that the conclusion does not depend on how the text was transcribed.

In conclusion, beyond the recipe collections (Chapter 26), medical and herbal texts close in content to the Voynich, in their original manuscript spelling, also avoid similar words next to each other. The Voynich property that both identical and one-glyph-different words cluster next to each other cannot be explained by the genre or the spelling variability of such texts of the period.

Source: Roussel, Klein, Dipper, Wegera, Wich-Reif (2024), Referenzkorpus Mittelhochdeutsch (1050–1350), Version 2.1 (CC BY-SA 4.0).

28. 15th-century herbals and medical books (ReF)

Following the Middle High German texts (up to 1350) of Chapter 27, we measured texts from around the 15th century, the period of the Voynich. The source is the Reference Corpus of Early New High German (ReF: Referenzkorpus Frühneuhochdeutsch 1350–1650, Version 1.0.2), which transcribes manuscripts and prints in their original spelling, including abbreviation marks and long s (ſ). From it we chose 10 medical, herbal, natural-history and divination texts. The core ones are the German herbal Gart der Gesundheit (Mainz, 1485), which gives an illustration and the virtues of each plant and was printed by Schöffer’s press, which had published the Latin Herbarius measured in Chapter 7 the year before, and a handwritten herbal and recipe collection of 1511. As counterparts to the Voynich’s star charts and astrological drawings we also added a moon divination book, Hartlieb’s Book of all forbidden arts and an astrological annual forecast.

Values are ratios to chance (words shuffled within each page); z is the deviation from chance (200 shuffles). Words are the space-separated units of the manuscript or print, with words split at line ends rejoined. “Normalised” changes ſ to s and removes abbreviation marks. Each text is measured over the portion transcribed in the corpus (up to 25,000 words).

• All 10 texts avoid putting similar words next to each other (adjacent 0.23–0.56×). Gart der Gesundheit is at 0.51× (z ≈ −-15), clearly below chance.
• “Same” reaches at most 0.56× and “1-glyph” at most 0.79×; none is above 1× on either. h2 is 2.95–3.28 bits, clearly higher than the Voynich (2.24).
• Neither manuscript versus print nor normalising the spelling changes the conclusion.

In conclusion, even in the genres closest to the Voynich’s drawings and from its own period (plant-by-plant herbals, recipe collections, surgery books, moon divination), texts in their original spelling avoid similar words next to each other. Across Chapters 26–28 we measured about 40 texts — 23 recipe collections, 13th–14th-century medical books and 15th-century herbals and medical books — and none has the Voynich property that both identical and one-glyph-different words cluster next to each other.

Source: Wegera, Solms, Demske, Dipper (2021), Referenzkorpus Frühneuhochdeutsch (1350–1650), Version 1.0.2 (CC BY-SA 4.0).

29. A 15th-century northern Italian recipe book and herbal

Chapters 26–28 centred on the German-speaking area, so we measured texts from northern Italy, a region that has also been linked to the Voynich. The first is a 15th-century Venetian medical recipe book in manuscript MA 400 of the Biblioteca Civica in Bergamo (first complete edition by Lemme 2024, CC BY), with 112 recipes of the kind “take …, grind it to powder, knead it with honey”. The edition fills in the manuscript’s abbreviations in round brackets, so we measured both “abbreviations kept” (the bracketed letters removed, close to how the manuscript looks) and “expanded”. Footnotes, page and folio numbers were removed. The second is the Italian herbal Herbolario volgare, printed in Venice in 1522.

Values are ratios to chance (words shuffled within each page); z is the deviation from chance (200 shuffles). *The Herbolario volgare text is the machine transcription from page images on the Internet Archive; it has reading errors (e.g. long s read as f) and includes the table of contents, so it is indicative only (first 34,859 words).

• The recipe book also avoids putting similar words next to each other (adjacent 0.51–0.62×). “Same” is 0.18–0.24×, particularly low.
• “1-glyph”, on the other hand, is 1.12–1.22×, the first text measured so far slightly above 1×. Most of it is the pair puo and tuo from the recipe formula “puo’ tuo’” (then take …; 11 times), so the count itself is small. Because identical words are clearly avoided, this is not the Voynich combination in which both identical and one-glyph-different words cluster next to each other.
• Herbolario volgare is at 0.54× and, like the other herbals, avoids similar neighbours (h2 3.42).

In conclusion, the Voynich combination does not appear in a 15th-century northern Italian recipe book or an early 16th-century herbal either. A formulaic recipe book can place rhyme-like one-glyph pairs next to each other, but it avoids repeating the same word — the opposite of the Voynich pattern.

Sources: Lemme, C. (2024). «Il ricettario veneziano del ms MA 400 della Biblioteca Civica “A. Mai” di Bergamo (cc. 66r-98v)». Quaderni Veneti 13, 31–70 (CC BY). Herbolario volgare (Venice, 1522), Wellcome Collection copy, Internet Archive.

30. Do head and tail preferences follow the type of drawing?

Head and tail preferences differ between sections (Chapters 12 and 14). But when the section changes, the hand and Language A/B often change too, so it was not clear whether the differences come from the content of the drawings (plants, baths, jars, etc.) or from the hand and the time of writing. If tail preferences were tied to the type of drawing, that would be a clue that tails carry meaning.

We therefore compared pages with different types of drawing within the same hand and language. Two such groups exist: Hand 1 / Language A has herbal and pharmaceutical (jar) pages, and Hand 2 / Language B has herbal, bath, text-only and cosmological pages. Pages were grouped into cells of “drawing type × quire” (a quire is a gathering of folded sheets bound together), and we measured how much the distributions of heads, tails and whole words differ between cells (JSD; 0 means identical). To remove the effect of different word counts, we drew 350 words at random from each cell and averaged over 200 draws.

“Halves of the same cell” is the difference when one cell is split at random into two, i.e. the size of chance fluctuation. p is the share of 2,000 permutations of drawing-type labels among cells in which the gap between “different drawing” and “same drawing, other quire” is at least as large as observed.

• Tail preferences are not tied to the type of drawing. For Hand 1, cells with different drawings differ (0.161) about as much as herbal cells from different quires (0.154; p = 0.34). For Hand 2, cells with different drawings are if anything closer (0.107; p = 0.82).
• The decisive factor is the quire. Where one quire contains pages with different drawings, their tails are especially close. In the bath quire, bath pages and text-only pages differ by 0.063, hardly more than two random halves of one cell (0.059). Herbal and jar pages in the same quire (0.115) are also closer than herbal pages in different quires (0.154).
• Heads show a weak difference only for Hand 1 (p = 0.03). But this is one of six tests, and herbal and jar pages in the same quire (0.191) are again closer than herbal pages in different quires (0.209), so it cannot be attributed to the type of drawing.

In conclusion, the differences in head and tail preferences are determined not by the content of the drawings but by which quire the text was written in (probably the time of writing or a continuous stretch of work). This fits “the sheet is the unit” of Chapter 14. At least with this measure, there is no evidence that word tails represent the type of drawing — plants, baths, jars. Quire differences are clearly visible with this yardstick (the same drawing type in another quire gives a larger value), so an effect of drawing type of similar size would have been detected.

31. Steering altered copies towards “forms used elsewhere”

One of the differences left in the final model of Chapter 25 (R7-2) was how rare words arise. Among the model’s once-only words, too many had a one-glyph-different word within the previous 10 words: too many new forms arose from altered copies of nearby words. We therefore added a mechanism: when a copied word is altered, the alteration is retried until the result is a form already written earlier (at most 20 tries; if none is found, the last attempt is used). The assumption is that the scribe, even when altering a nearby word, often changed it into a form used elsewhere.

“Origin of rare words” is the share of once-only words with a one-glyph-different word within the previous 10 words (ratio to chance). Values for the search seeds (101–106) are means over three different “orderings” (see below). The hold-out seeds (201–206) were not used in the search.

• The origin of rare words moves close to the manuscript. It goes from 1.77 to 1.40, closing about 83% of the gap to the manuscript (1.32). With the hold-out seeds it goes from 1.86 to 1.29, almost the manuscript value. With at most 5 tries, steering half of the words gives 1.63 and steering all gives 1.46: the stronger the steering, the closer to the manuscript.
• Other properties hardly change. h2, the share of once-only words, page character and similarity to the word directly above stay within the ranges of the four key targets. In exchange, the number of word types drops slightly (2,850 → 2,728; manuscript 2,889) and one-glyph neighbours rise slightly (1.23 → 1.27; manuscript 1.21).

A correction on evaluation noise. While checking this chapter we found that the generated text also changes with Python’s set ordering (PYTHONHASHSEED). For the same R7-2, J on the search seeds ranges from 0.070 to 0.154 across orderings, and similarity to the word directly above is 1.21, 1.25, 1.28 (just outside the target range for some orderings). This ordering was not fixed in the autonomous loop of Chapter 24, so differences in J between candidates of up to about 0.05 should be read as meaningless. The steered version fluctuates less (J 0.081–0.088) and meets all four key targets under all three orderings.

In conclusion, the mechanism “when altering a copied nearby word, change it into a form used elsewhere” almost removes the remaining difference in how rare words arise (Chapter 25). The statistics fit well if the scribe, while taking cues from nearby words, mostly chose forms from among words already in use. This version (R7-2 plus att_p = 1.0, att_T = 20) is the model closest to the manuscript so far.

32. Discrimination test: can a classifier tell the manuscript from the model?

So far we have compared the manuscript and the model on measures we chose ourselves, which cannot reveal differences in properties we did not choose. We therefore mixed manuscript pages with generated pages (same word count and line shape per page) and asked a classifier (logistic regression) to tell them apart. It uses about 300 features per page: glyph frequencies, glyph pairs, word lengths, word-initial and word-final glyphs, line-initial and line-final glyphs, junctions between words, and repetition and vocabulary measures. We used the 193 pages with at least 50 words and 5-fold cross-validation, always keeping a real page and its generated version on the same side. The score is AUC: 0.5 means indistinguishable, 1.0 means perfectly distinguishable. Generation used seeds not used in the search (201–203).

“All” uses all features; the other columns use only that kind of feature (AUC, mean of 3 runs). The control, “manuscript shuffled within pages”, only breaks word order and is perfectly distinguished through line edges and junctions (confirming that the classifier is powerful enough).

• At Chapter 31 the model was still clearly distinguishable (AUC 0.90). The classifier relied most on word length. Manuscript words vary little in length (SD 1.65), and only 0.9% have 9 or more glyphs; the model had 3.2% (SD 1.91), far too many overlong words. Building words glyph by glyph lacked a “word-length habit”. Adding a habit of choosing lengths that match the manuscript’s length distribution — both when building new words and when choosing which word to copy — made word length useless to the classifier.
• The next cue was the excess of words starting with q. In the manuscript 15.2% of words start with q; in the model 21.9%. The cause: when choosing a source word by how well it fits the link to the previous word, words starting with the very common qo were favoured regardless of the link (12.1% after endings other than y; manuscript 7.2%). Discounting the fit by how common the start is lowered q-initial words to 18.8%. At the same time y→q, a gap left since Chapter 25, reached 3.70× (3.66× with hold-out seeds), almost the manuscript’s 3.81×. y→q was weak not because q was too rare after y, but because q was too common after other endings.
• With these two fixes and small adjustments (stronger family attraction for source words, slightly more unaltered copies), AUC fell from 0.90 to 0.71. The model is still more distinguishable than chance (0.5). The main remaining cues are the excess of adjacent one-glyph-different pairs and the choice of word after words ending in o.

Main measures of the Chapter 32 version (search seeds 101–106 and hold-out seeds 201–206, 6-run means). With the hold-out seeds, page character is 1.68, slightly above the target range (1.60 ± 0.08).

In conclusion, the discrimination test found two differences invisible to the measures we had chosen: a “word-length habit” and a “bias towards common word starts”. Fixing both brought the model closer to the manuscript while keeping the other measures, and also removed the long-standing y→q gap. It is natural to think that the scribe, even when building words glyph by glyph, had an approximate length in mind (or fitted glyphs into slots of fixed length).

33. Comparing with a meaningful ciphertext: the Naibbe cipher

Greshko (2025) proposed the “Naibbe cipher”, which turns Latin or Italian into Voynich-like strings using only tools available in the 15th century (playing cards and dice). The plaintext is cut into pieces of one or two letters, and each is replaced, using one of six tables chosen at random, by a form resembling manuscript words (in EVA). It is a decipherable, “meaningful” ciphertext. With the published implementation (github.com/greshko/naibbe-cipher) we encrypted about 35,000 words each of Pliny’s Natural History book 16 and Caesar’s Gallic War (Latin) and Dante’s Divine Comedy (Italian), and measured them with the same yardstick as the manuscript. The ciphertext was laid out with the same word counts and line shapes as the manuscript pages.

Ratios to chance (words shuffled within each page) etc., defined as in earlier chapters. Distinguishability is the AUC of the classifier of Chapter 32 (0.5 means indistinguishable). The Naibbe cipher has no mechanism for line or page structure (as its author states), so we also checked discrimination without line-start and line-end features.

• Glyph predictability is the Naibbe cipher’s strength. h2 is 2.16–2.19, almost matching the manuscript (2.24). This confirms that the manuscript’s “predictable glyph sequences” can arise from a meaningful plaintext.
• On the other hand, none of the context properties appear. Adjacent similarity is 0.86–0.90×: like ordinary text, similar words are avoided. y→q is 0.93–1.00× (manuscript 3.81×), similarity to the word directly above 0.99–1.05×, and page character 1.00–1.02: all “no linkage” values. The share of once-only words, 39%–44%, is also much lower than the manuscript’s (68%).
• The classifier distinguishes it perfectly (AUC 1.00). Without line-start and line-end features it is still 1.00, and nearly every kind of feature suffices. The Chapter 32 model scores 0.71 (0.79 on Language B pages only).

Note that this is not a fair comparison. The Chapter 32 model is built from the manuscript’s own statistics (glyph continuations, section biases, etc.), whereas the Naibbe cipher uses its own tables and carries a meaningful plaintext, and its author states that line and page properties are outside its scope. Still, one thing follows: a cipher of the kind “replace each word independently” does not produce the manuscript’s most distinctive features — identical and one-glyph-different words clustering next to each other, the next word’s start depending on how the previous word ends, and nearby lines and the same page being similar. If the manuscript is a ciphertext, something beyond the substitution must have been added: a procedure of writing while looking at nearby words (copying with alterations), or of choosing the substitution to fit the previous word.

Source: Greshko, M. A. (2025). The Naibbe cipher: a substitution cipher that encrypts Latin and Italian as Voynich Manuscript-like ciphertext. Cryptologia. doi:10.1080/01611194.2025.2566408. Implementation and data: github.com/greshko/naibbe-cipher (modified MIT licence). Encryption used the default settings (52-card deck, standard respacing).

34. Can a “scribal habit” be added while keeping the meaning?

The Naibbe cipher of Chapter 33 had none of the context properties. But it has freedom: for the same one or two plaintext letters there are 6 to 36 possible substitutions, and any of them decrypts to the same plaintext. We therefore chose among these candidates with scribal habits. Whatever is chosen, the decryption is unchanged, so the meaning is kept. There are three habits:

• Leaning towards nearby words: prefer forms identical to, or one glyph different from, words visible on the same line and the previous three lines.
• Forms often used on the page: prefer forms already used several times on that page.
• Link to the previous word: prefer starts that fit how the previous word ends (the same choice as in the Chapter 32 model).

We varied the strength of the habits for the three works by Pliny, Caesar and Dante (means over the three).

“No habit” chooses according to the card weights (almost the same as the ciphertext of Chapter 33, but values differ slightly because we choose by weight rather than drawing from a deck). Decrypting the strong-habit ciphertext with the author’s decryption program returned the original plaintext for 97.9% of tokens, and the remaining 2.1% included the correct plaintext as one of two candidates (the no-habit version has the same kind of ambiguity for 1.1%). No token decrypted wrongly.

• About half of the context properties could be produced while keeping the meaning. Adjacent similarity went from 0.87 to 1.09, one-glyph neighbours from 0.95 to 1.21 (manuscript 1.21), similarity to the word directly above from 1.00 to 1.26 (manuscript 1.28), page character from 0.98 to 1.17, and y→q from 0.95 up to 1.85.
• But “the same word next to itself” cannot be produced. The rate of identical neighbours stays below 1× even with strong habits (0.69 → 0.79; manuscript 1.34). Each substitution candidate belongs to a different plaintext piece, so the same word can be written twice in a row only where the plaintext itself repeats. The share of once-only words stays at 42% (manuscript 68%), and stronger habits move glyph predictability away from the manuscript (2.24): 2.09 → 1.90.
• The classifier distinguished every setting perfectly (AUC 1.00), because the Naibbe tables themselves produce word shapes unlike the manuscript’s.

In conclusion, even a meaning-preserving cipher can, with a habit of choosing forms similar to nearby words, produce one-glyph-different neighbours and similarity to the line above. But the manuscript’s most conspicuous feature — repetition of the same word (sequences like daiin daiin, qokedy qokedy) — does not arise in this kind of cipher unless the plaintext repeats in the same places. If the manuscript is a ciphertext, either the plaintext itself was highly repetitive, or some of the repeats are meaningless words (skipped in decryption). This differs from the simple form rejected in Chapter 5 (“specific filler words were mixed in”): here the fillers would enter as a habit of copying nearby words.

35. What the repetition of identical words really is

Chapter 34 showed that the manuscript’s “same word next to itself” is hard to produce with a meaning-preserving substitution cipher. So what does this repetition resemble? We examined the 287 places where the same word follows itself on a line (1.31× chance) and compared them with Indonesian, which has grammatical reduplication (repeating a word to mark plurality, as in orang-orang “people”), and with the Chapter 32 model (a mechanism of copying nearby words).

“Repeat ratio” is the rate of the same word following itself on a line, relative to chance (words shuffled within each page, 30 times); the frequency and position columns are likewise ratios to chance. “Next word also similar” compares how often the word right after a repeat is identical to or one glyph different from the repeated word, against non-repeat positions. “Triples” compares how often a third copy follows two, against the overall repeat rate. The Bibles were laid out with the manuscript’s page and line shapes, so their line-position values are meaningless. Texts with fewer than 50 repeats are shown as “—”. Indonesian reduplication is split at the hyphen and counted as two words.

• Not a grammar of particular words. Indonesian repeats concentrate on specific nouns such as “people” and “disciples” (orang 201, pengikut 142, anak 32, laki 29, guru 29), and each is complete in two words. The words most often repeated in the manuscript (chol 24, qokeedy 18, qokedy 15, qokeey 12, daiin 11, ol 9, shedy 9, chedy 9) are ordinary, common manuscript words of average length (0.98× the mean). The 20 most frequent words repeat only at the chance rate (0.98×), while mid-frequency and rare words repeat more readily (1.80× and 3.06×).
• Line edges are avoided. Repeats right after the line-initial word are 0.44× chance, and the line-final pair 0.77×; repeats gather in mid-line (1.59×).
• Repeats occur as part of runs of similar words. The word right after a repeat is identical to or one glyph different from the repeated word 3.31× more often, and a third copy is 4.5× as likely (sequences like qokedy qokedy qokeedy). Indonesian reduplication has none of this. The Chapter 32 model, which copies nearby words, does (2.52× and 3.4×).

In conclusion, the manuscript’s repetition of identical words is quite unlike grammatical reduplication and fits well the trace of a way of writing: “looking at nearby words and writing the same word, or a slightly altered one, in succession”. Together with Chapter 34, this makes it likely that the repetition does not carry the meaning of a plaintext but arises as a writing habit (or as meaningless words). The Chapter 32 model differs at line starts (line-start pair 1.01×, i.e. at chance; line-end pair 0.33×). The scribe seems to have avoided repeating a word right at the start of a line — the next improvement for the model.

36. An autonomous search aimed at the discrimination test (second night)

The Chapter 32 model could still be told apart by the classifier (AUC 0.71). So we ran the autonomous improvement loop for another night under the same rules as Chapter 24. The goal was to bring the classifier’s distinguishability (AUC) towards 0.5, with a penalty added whenever a key measure tuned so far (h2, once-only words, page character, directly above, y→q, identical and one-glyph neighbours, rare-word origin) left its target range.

• Every judgement is a six-run mean; following Chapter 31, the Python set ordering was also changed from run to run.
• Promising candidates were measured again with six other seeds and compared with the best on the mean of both (so as not to adopt lucky draws).
• Six candidates form a block; after two blocks without improvement, the loop examined which kinds of cues the classifier relied on and narrowed the search to the related knobs.
• A watchdog restarted the search automatically if it stopped. New mechanisms we thought of were injected as candidates and evaluated in the same way.

From 17:09 until 6:30 the next morning, 107 candidates were evaluated and 12 adopted. The full history is in disc_history.csv, the search log in disc_search.log, and the final settings in final_params_ch36.json.

The main adopted changes and the distinguishability at that point (search seeds). “Earlier classifier” is the one from Chapter 32 (logistic regression on about 300 features). “Strong classifiers” use about 800 features (adding glyph triples and the frequencies of the 200 most common words) and average two methods, logistic regression and gradient boosting (stacked decision trees). The judge was switched to these partway through.

First half: word-shape habits make the earlier classifier fail

The cues the classifier relied on showed mismatches in word starts and ends, such as too many words starting with q and ending in o. We therefore added weights that favour shapes matching the manuscript’s distributions of word length, first glyph, last glyph and glyph pairs (a shape habit). New words are built as several candidates and chosen by this weight, and the weight is also applied when choosing a word to copy. With this, the earlier classifier’s AUC fell from 0.75 to 0.50: no better than chance.

A pitfall: a different classifier could still tell them apart

We then checked with stronger classifiers. Their AUC was 0.81 (gradient boosting 0.87), still clearly distinguishable. The model had only been fitted to what one particular classifier looks at; it could not be said to have come closer to the manuscript. So partway through the search the judge was switched to the strong classifiers.

Second half: a habit for frequent words

The strong classifiers’ cues showed mismatched frequencies of common words (the one-glyph words s and r too rare, o, dain and ain too common, etc.). Adding a weight matching the frequencies of the manuscript’s 300 most common words lowered the strong classifiers’ mean AUC from 0.81 to 0.65. Logistic regression can no longer tell them apart, but gradient boosting still can (0.79).

Check with seeds not used in the search

Six-run means for the Chapter 32 and 36 versions with seeds 201–206 (and orderings different from the search).

• Earlier classifier: 0.76 → 0.50; indistinguishable from chance.
• Strong classifiers: logistic regression 0.85 → 0.50, close to chance; gradient boosting 0.90 → 0.80, still clearly distinguishable.
• Key measures: all within the target ranges.

What gradient boosting sees. Feature importances show that the share of one-glyph words matters most. Its overall mean is almost the same as the manuscript’s, but its spread across pages differs: in the manuscript one-glyph words appear moderately on every page, while in the model pages split into those with none and those with many (standard deviation of the per-page share: manuscript 0.026, model 0.045). The share of adjacent one-glyph pairs and the variety of words within a page show the same tendency. Because the model copies nearby words, the same forms tend to clump within a page, whereas the manuscript’s words are spread more evenly over the page.

Caution. The shape habit and the frequent-word weights added in this chapter are built from the manuscript’s own distributions. This amounts to giving the model the scribe’s preferences for words and shapes, and part of the reduced distinguishability is the effect of that fitting. It moves somewhat away from the Chapter 24 goal of “simple rules that could be carried out by hand”.

In conclusion, on statistics taken one at a time, such as glyph sequences and word frequencies, the model and the manuscript can no longer be told apart. The remaining difference lies in combined statistics of how words are spread over a page, showing that the model’s central mechanism, copying nearby words, clumps words more than the manuscript does. A natural next step is a different way of writing that keeps adjacent similarity while copying less (for example, choosing a set of candidate words before starting a page).

37. Two Claudes compared: one night from the same starting point

We handed the continuation of Chapter 36 to two Claudes at the same time, under the same conditions: Opus 5.5, which has been writing this research note, and Sonnet 5.5 (reasoning effort set to "medium").

• Same material. Both received the same handoff package: the Chapter 36 model and code, the transcription, and this research note so far.
• Same rules. Every judgement is a 6-run mean, and promising candidates are re-checked on other seeds. After two blocks without improvement the approach must change, and the deadline was 6:30 JST.
• Same yardstick. The goal was to bring the strong classifiers' AUC (Chapter 36) towards 0.5 while keeping the key statistics within their target ranges. The classifiers, the objective and the targets were not to be changed.

In the morning we graded both with a final exam that neither had seen. It uses seeds 401–406, which were never used in the search, and five classifiers. Two of them, marked *, were never used in the search: a random forest, and gradient boosting with added features for how words are spread within a page. The exam was written before the search began and was not changed afterwards. Note, however, that the Opus side wrote it.

What happened

In the afternoon both work environments were restarted midway, and the search processes stopped.

• Opus was checking every ten minutes, so both times it restarted from the saved best and kept going until 6:30 the next morning.
• Sonnet could not resume after its conversation was compacted, and stopped at 17:26 on the 18th.

Because the search times differ, we also put Opus's best as of 17:26 through the same exam, to compare equal amounts of time.

Mean AUC over six runs in the final exam (seeds 401–406); 0.5 means the classifier cannot tell them apart. * marks classifiers never used in the search. Candidates are settings evaluated, not counting re-checks.

• At equal time (about 8 hours), Sonnet was ahead. Gradient boosting gave 0.65 for Sonnet against 0.70 for Opus, and the random forest 0.61 against 0.68. Sonnet evaluated fewer candidates, but spent its time diagnosing what the classifier relied on, and found a mechanism that worked.
• The final Opus model scored lowest. Gradient boosting gave 0.59 and the random forest 0.50. What helped over the night's 13 extra hours was a weight on three-glyph sequences, together with parameter search.
• The ranking is the same on the classifiers never used in the search. Neither improvement is just a fit to the search classifier.

Key statistics in the same exam. Bold marks values outside the target range.

The key statistics stayed mostly within range. In both searches, however, the share of once-only words fell slightly from the starting point (0.663) to just outside the range. Identical neighbours also fell just below the range for Sonnet (1.24) and for the final Opus model (1.22). At 17:26, Opus's rare-word origin (1.47) was slightly outside.

What both searches found independently

• Don't copy one-glyph words. At handoff, the biggest cue was how much the share of one-glyph words varied from page to page. Both changed the model so that one-glyph words are no longer copied from nearby. Sonnet picks them directly from rates by section and position in the line; Opus writes them fresh instead of copying.
• Random per-page shifts in preference don't help. Both tried this, and neither improved.
• What remains is in combinations of features. Sonnet found that the spread of per-page features along their principal components is 20–30% smaller in the model than in the manuscript. Opus checked that, taken one at a time, every feature differs very little (single-feature AUC 0.42–0.58). The wording differs, but the conclusion is the same.

Where they differed

• Sonnet borrowed the profile of another page. It takes a little of the glyph, first-glyph, last-glyph and length proportions from another page in the same section (never the page being generated) and uses them to choose new words. Instead of shifting preferences at random, it borrows the way real pages actually vary. It also found and fixed the reason the same settings and seed gave different results each time (an unseeded random number generator).
• Opus refined the shape weights. It pushes altered copies towards forms that occur anywhere in the manuscript, and adds three-glyph sequences to the shape weights. It also tried per-section shape corrections and within-page balancing, but neither was adopted.

Caveats

• Values below 0.5. The strong logistic regression fell below 0.5 for both. This is not "telling them apart the other way round"; it is a bias that page-wise cross-validation tends to show when the two groups are almost indistinguishable. Because the objective treated lower as better, values below 0.5 were also counted as improvements. That is a flaw in the yardstick.
• This did not compare the models' raw ability. The effort settings differed, and a chance event — the environment stopping — shaped the result a great deal. What was compared is how each ran a night of autonomous work, and whether it recovered when that work stopped.

In conclusion, at equal time Sonnet had built the better model. The final difference came down to whether, when the long run stopped midway, each noticed and restarted it by itself. Gradient boosting is still at 0.59, short of 0.5. The two searches reached the same wall — combinations of features — by different mechanisms, so the next step is worth trying both mechanisms together.

38. The second comparison, and a "leak" in the evaluation

After Chapter 37 we gave the same two Claudes (Opus 5.5 and Sonnet 5.5) another night under identical conditions. There were three changes from Chapter 37:

• Starting point. Both started from Opus's final model from Chapter 37, which had scored best in the final exam.
• Objective. The objective now measures the distance from 0.5 rather than the AUC itself. Chapter 37 had shown that the old objective counted values below 0.5 as "better".
• Hourly progress reports. These were made mandatory: every hour each run checks that its search is still alive and restarts it if not. Last time, what decided the outcome was whether a run recovered when its environment stopped.

This time neither run stopped, and both sent hourly reports until morning. Sonnet stopped its own search a little before the deadline, re-measured its top candidates on other seeds, and re-chose its final setting. The final exam was the same as in Chapter 37: seeds 401–406 and five classifiers, where * marks those never used in the search.

Mean AUC over six runs in the final exam; 0.5 means the classifier cannot tell them apart. Candidates are settings evaluated.

On this table alone Sonnet improved most, with gradient boosting at 0.56 and the random forest at 0.48. Opus's gradient boosting (0.58) is about the same as the start (0.59), and its random forest got worse (0.50 → 0.56). However, these numbers include a leak in the evaluation.

The leak

Both searches adopted the mechanism Sonnet built in Chapter 37, borrowing another page's profile: each model page borrows a little of the glyph, first-glyph, last-glyph and length proportions of another manuscript page in the same section.

• Why it leaks. Model page k then carries the profile of manuscript page j. The classifier is evaluated by splitting pages into training and test sets by page number. When manuscript page j is in the test set, model pages that borrowed j's profile can stay in the training set, labelled "fake". The classifier learns that "pages like j are fake" and judges the real page j as more fake-like. Distinguishability then comes out lower than it really is.
• How we checked. We re-evaluated after removing from training the model pages that borrowed from manuscript pages in the test set. To see the effect of the removal itself, we did the same for the starting point, which borrows nothing.

Strong-classifier AUC on the final-exam seeds (401–406), normal evaluation versus evaluation with the leaking pages removed from training; means over six runs. Opus's code still has an unseeded random generator in generation, so these are different generations from the first table and can differ by about 0.02.

• Effect of the removal itself. For the starting point, which borrows nothing, removal barely changes gradient boosting (0.60 → 0.61).
• Borrowing versions. Gradient boosting clearly rises: Sonnet 0.56 → 0.61, Opus 0.61 → 0.64. Sonnet's leak is larger, probably because this time it also borrowed the page's rate of repetition from the same page.
• Leak-free version. Opus noticed the leak during the search and built a version that, instead of borrowing a specific page, draws a profile at random from how pages vary across the whole manuscript. Removal does not change it (0.60 → 0.59), and it sits at the same level as the start.

So with the leak removed, neither search lowered gradient boosting below the start (about 0.60), and Opus's final model is actually worse. Chapter 37's Sonnet final also contained the mechanism, but its leak was small (0.65 → 0.67), so the conclusion of Chapter 37 stands.

The regression falling below 0.5 also happens for the starting point, which borrows nothing (0.39). This is a separate bias that cross-validation tends to show when two groups are nearly indistinguishable. Sonnet also confirmed this property independently.

Key statistics in the same exam. Bold marks values outside the target range.

What each found

• Opus: why once-only words fall short. The missing once-only words are short. Among words of 2–4 glyphs, the share occurring once is 0.36, 0.41 and 0.53 in the manuscript but 0.21, 0.33 and 0.43 in the model. The cause is that new words are chosen from candidates by the shape weights too strictly, so common forms keep winning. Loosening that choice brings once-only words back without disturbing glyph predictability (h2), and greatly reduces the penalty for the key statistics. However, it also lets in more long words, which the classifier detects, so it is not in the final setting.
• Sonnet: what drives identical neighbours. The shortfall in identical neighbours depends not on how often copies are altered but on how often nearby words are copied unchanged. Sonnet also confirmed a trade-off: choosing rare words as copy sources adds identical neighbours but removes once-only words.
• Both: copying too much from the word just before. Comparing features one at a time, both found that identical immediate neighbours are too frequent and identical words a little further apart too rare. Both then arrived at the same point: model pages have too few distinct words.

Summary

In the second round both runs searched until morning without stopping, and the progress-report mechanism did its job. But most of the drop in distinguishability came from the leak that profile borrowing brought into the evaluation. With the leak removed, gradient boosting has not moved from the Chapter 37 final (about 0.60).

From now on, any mechanism that uses statistics of manuscript pages has to be evaluated with training and test split in a way that accounts for which pages it drew on. The remaining difference is that model pages have too few distinct words, a point the two searches reached from different directions.

39. A rebuilt yardstick and a third night: five machines

Looking again at the evaluation leak found in Chapter 38, it turned out not to come only from profile borrowing. For a long time the model has built new words from the statistics of nearby pages. So in the old evaluation, which splits training and test by page, the statistics of a manuscript page sent to the test set reached the training side through the neighbouring fake pages.

• New evaluation. Consecutive pages are grouped into blocks of eight, and training and test are split by block. Fake pages within five pages of a test page are then removed from training. The classifiers and features are unchanged.
• Re-measured. For the Chapter 37 final, gradient boosting rose from about 0.57 to about 0.69, and the regression from about 0.38 to about 0.55. The indistinguishability figures since Chapter 32 were therefore too optimistic across the board. The regression values below 0.5 were also mainly due to this leak.

With this yardstick we ran a third night on five machines.

• Explorers: Opus 5.5 and Sonnet 5.5 (both in new conversations, reasoning effort "high"). Under the same rules as Chapter 38, they freely try new mechanisms.
• Measurers: three Haiku 5.5 instances (effort "medium"). Without changing any code, they measure a fixed table of combinations from end to end: 72 combinations of four settings — selection strength, the share of nearby words copied unchanged, the copy-distance table and the randomly drawn page profile. In a second phase they re-measured the area around the top results more finely, on three sets of seeds.

The work environment was restarted twice during the night, but every machine noticed at its hourly check and restarted by itself. One Haiku noticed that its measurement died together with its waiting command, and found the fix of starting the measurement detached; we passed this fix on to the other two.

Final exam (second edition)

The final exam was rebuilt with the same split as the new evaluation (seeds 401–406, five classifiers; * marks classifiers never used in the search). The classifiers and features are the same as in the Chapter 37–38 exams; only the way training and test are split differs.

Mean AUC over six runs in the final exam (second edition, leak-free split). 0.5 means the classifier cannot tell them apart.

• The final Opus model became the hardest to distinguish for every classifier. Gradient boosting fell from 0.70 to 0.63. On the two classifiers never used in the search, the random forest fell from 0.65 to 0.62, and the one with dispersion features from 0.70 to 0.63. The differences are clearly larger than the run-to-run spread (standard deviation 0.01–0.02), and they are not just a fit to the search classifier.
• Sonnet's final gives gradient boosting 0.67 and random forest 0.65, a little better than the start. Its regression, 0.49, is the closest to 0.5.
• The top of the Haiku grid gives gradient boosting 0.65, better than the start. However, in Haiku's own second-phase re-measurement (table below) the same setting came out the same as the start. The result depends on the set of seeds, so the effect of this setting is unclear.

Checking on seeds not used for selection

Choosing the best of many candidates makes it look better by whatever luck it had (the winner's curse). Before the deadline, both explorers re-measured the start and their final on seeds not used in the search (113–118).

Values as given in each explorer's report. O is the objective (0.5 + mean distance of the two strong classifiers from 0.5 + penalty); lower is better.

• Opus: on unused seeds too, O improved from 0.662 to 0.596. Gradient boosting fell from 0.680 to 0.645, and the penalty from 0.047 to 0.008.
• Sonnet: on the search seeds O fell from 0.657 to 0.627, but on unused seeds it was 0.662 against 0.646, with gradient boosting almost unchanged (0.680 against 0.675). Sonnet itself reported that the difference was within seed-to-seed variation and that the real goal was not reached.

The Haiku grid: what exhaustive measurement showed

• Phase one (means over 48 settings). Only a selection strength of 0.8 lowered gradient boosting a little (0.674, against 0.690 at 1.0). The share of nearby words copied unchanged moves the identical-neighbour ratio almost linearly (0.15: 1.23, 0.28: 1.33, 0.40: 1.41), matching the manuscript (1.34) at 0.28. The copy-distance table had no effect.
• Phase two (re-measured on three seed sets). The phase-one winner (strength 0.8, keep 0.28) came out the same as the start on the third seed set. Every setting falls within 0.65–0.67 on the three-set mean, a difference from the start within seed-to-seed variation. The phase-one winner was the lucky pick out of 72.

Haiku phase two: O (lower is better) for combinations of selection strength (sp) and keep share (lk), on three seed sets.

Exhaustive measurement confirmed that recombining existing settings does not give a reliable improvement over the start. In the explorers' runs too, the reliable gains came from new mechanisms.

What worked: Opus's diagnosis

• What the classifier sees is the direction of page-to-page variation. Taken one at a time, feature means almost match the manuscript. But along the principal components of manuscript pages (directions in which pages vary together), model pages vary only 0.5–0.7 times as much as the manuscript in the leading directions, and 1.3–1.5 times as much in the minor directions. One number per page — how far it lies out along the top 12 directions — gives the same AUC of 0.68 as the classifier.
• Likely cause. The model builds words from the average of the two neighbouring pages, which dilutes each page's own variation. With a neighbour correlation of 0.3, the variance becomes (1 + 0.3)/2 ≈ 0.65 times, which matches the observation.
• The fix that worked. Adding coherent variation drawn at random from how pages vary across the whole manuscript (no specific page is borrowed, so no leak). Removing just this part worsens O by 0.073; most of the night's improvement came from it.
• How words repeat. The manuscript hardly repeats rare words within a page, while frequent words repeat nearby (2–10 words later). The model repeated rare words 8–15 times too often. Adding this property brings once-only words in line with the manuscript and halves the penalty for the key statistics.

Sonnet reached the same problem from another direction: in the manuscript, adjacent pages' glyph proportions are about twice as correlated as in fake pages, and the fewer words a page has, the easier it is to distinguish (AUC 0.78 below 80 words, 0.64 above 200). But the four new mechanisms it built to fix this did not move gradient boosting.

Key statistics in the same exam. Bold marks values outside the target range.

Caveats

• The new yardstick may still be generous. Manuscript pages outside the five-page buffer remain in training; if fake pages resemble them, distinguishability comes out lower.
• Fitting to manuscript-wide statistics. The randomly drawn profiles use how pages vary across the whole manuscript (principal directions and spreads). They are not tied to any specific page, so there is no leak, but they include a fit to the manuscript as a whole.
• Messages during the run. At 21:24 we sent the same summary of Haiku's phase one to Opus and Sonnet. Both tried it, and neither adopted it. Opus also received one unrelated message at about 15:40.

In short, rebuilding the yardstick showed that the model is easier to distinguish than we had thought. On that yardstick, adding coherent page-to-page variation and a writer-like way of repeating words clearly lowered distinguishability. What remains lies in the finer details of the direction of page variation.

40. Checking the yardstick: splits and the amount of training

Chapter 39 listed, among its caveats, that the new yardstick might still be too lenient. Before the next overnight search we checked this. We looked at two things.

• The split. How much do the values move when we change how pages are split into training and test (block size and buffer width)?
• The amount of training. How much do they move when we change how many examples the classifier sees?

The material is the fake pages made by the start (the Chapter 37 final) and by the Opus final (Chapter 39). They were generated with the same seeds as the final exam (401–406), and we averaged over the six runs. These seeds were used only to compare yardsticks, never to choose a model or setting. Features and classifiers are those of the final exam (gradient boosting, random forest and regression on the strong features).

Changing the split

At first we simply widened the buffer and measured again. The values fell, but only because fewer pages were left for training. With a 15-page buffer on the fake side, for example, only about 25 fake pages per fold remain for training, against 154 manuscript pages. In that setting we cannot tell whether a lower value means a leak was removed or the classifier simply learned too little.

So for every split we drew the same number of manuscript and fake pages at random for training (100 each, and 55 each; three draws, averaged).

Distinguishability (AUC) for each split, mean over seeds 401–406. GB is gradient boosting, RF is random forest. "On both" also removes manuscript pages near the test pages from training. "5 contiguous parts" cuts the manuscript into five consecutive parts. Compare rows within the same training size (values are lower overall in the 55-page rows).

• The buffer on fake pages hardly matters now. With training size held equal, no buffer and a five-page buffer give almost the same values (gradient boosting: start 0.65 vs 0.65, Opus 0.60 vs 0.61). The leak found in Chapter 39 seems to be blocked mainly by splitting in blocks of eight pages.
• Leniency from nearby manuscript pages is real but small. Adding a five-page buffer on the manuscript side as well raises the values a little (random forest: start 0.65 → 0.68, Opus 0.61 → 0.64). For gradient boosting, Opus barely changes (0.61 → 0.61); the leniency feared in Chapter 39 is about 0–0.03, depending on the classifier.
• Coarser splits show no sign of a hidden leak. Blocks of 16 pages or buffers of 10–15 pages move the values by only about 0.01. Splitting into five contiguous parts with buffers of 10–30 pages even lowers them slightly. This is probably because the writing of the sections in the test part is then missing from training.
• The ranking does not change. For every split, the Opus final is harder to distinguish than the start.

Changing the amount of training

Comparing the two halves of the table, going from 55 to 100 training pages of each kind raises the values by 0.04–0.07 under the same split. The amount of training mattered far more than the split.

Manuscript pages cannot be increased (193 pages have 50 words or more, and about 154 per fold are available for training). Fake pages, however, can be generated in any number by changing the seed. So, keeping the current yardstick (blocks of eight, buffer of five on the fake side), we changed the amount of training in two directions.

• Less. Thin the training pages to 35–85%.
• More fake pages. Add fake pages made with other seeds (701–708, used nowhere else) to training, up to eight times as many. Because manuscript and fake pages are then unbalanced, the classifiers were trained with the two classes weighted equally. Fake pages within five pages of a test page are removed from the added pages too.

The same values as the figure (mean of six runs). The left column is the number of fake training pages per fold. The regression barely moves at any amount.

• At the current amount, the classifiers have not finished learning. For the Opus final, gradient boosting gives 0.64 at the current amount, 0.71 with four times the fake pages, and 0.73 with eight times. Beyond four times the rise slows, but it is still creeping up at eight times. The start also rises, from 0.69 to 0.73 (4×) and 0.75 (8×).
• The gap shrinks but does not close. The difference between the start and Opus was 0.047 (gradient boosting) at the current amount, 0.018 at 4× and 0.019 at 8×.
• The regression stays in the 0.5s however much it is trained. A view that adds features up one by one sees almost no difference any more. What remains lies in combinations of features.

What this means

• Final-exam values understate the true distinguishability. The exam measures distinguishability when the classifier is trained on about as many fake pages as there are manuscript pages. Trained on eight times the fake pages, gradient boosting gives 0.73 even for the Opus final.
• Part of Opus's improvement came from being "hard to learn from few examples". The Opus setting adds random coherent variation to each page. That spreads the fake pages out, so the cues are harder to learn from few examples. With more examples, that part of the advantage fades. But the gap remains even at eight times, so the improvement was not all an illusion.
• The next search should be judged by a classifier trained on more fake pages. The search already generates fake pages for six seeds. When testing one seed's fake pages, adding the other five seeds' pages to training gives a "six-times" classifier at no extra generation cost. The final exam will also gain a version with extra fake pages from exam-only seeds.

Caveats

• The manuscript side cannot be enlarged. The slowing rise may simply reflect the limited number of manuscript examples (about 154 pages). The true distinguishability may be higher still.
• Class weighting. The classifiers trained on extra fake pages weight the two classes equally. At the current amount they give almost the same values as the final exam (gradient boosting: start 0.69 vs exam 0.70, Opus 0.64 vs exam 0.63).
• Classifier settings were not changed. Stronger classifiers with more or deeper trees were not tried.

In short, no large leak remains in the Chapter 39 yardstick. But the classifiers were shown too few fake pages, which made the models look harder to distinguish than they are. With eight times the fake pages, even the Opus final is distinguished at 0.73, and the difference from the manuscript is still clear.

41. Seven critical editions provided by a specialist

Chapter 29 measured a 15th-century Venetian recipe book and a herbal printed in 1522. We wanted more handwritten medical texts from 15th-century northern Italy, but the Italian corpus of medieval medical texts (ReMediA, OVI) was unreachable because of a server problem. We therefore wrote to Prof. Ilaria Zamuner (University of Chieti-Pescara), who co-directs ReMediA. She kindly provided seven critical editions prepared by herself and her colleagues, and agreed that we may publish the values and cite the editions. The texts and PDFs themselves are not published.

• What was measured. Besides recipe books, we included surgical and medical treatises written mostly in continuous prose. Prof. Zamuner suggested this, because recipe books are full of set formulas.
• Method. As in Chapter 29. From the edited text we removed notes, numbering and folio marks, and measured the text with abbreviations expanded. Texts longer than the Voynich (34,859 words) were cut into sections of that length; the table shows the mean over sections.

Ratios to chance ("adjacent" = share of words with a word differing by at most one glyph 1–2 positions earlier, split into "same" and "1-glyph"). z is the deviation from chance (within-page shuffle). For long texts, the small numbers under "adjacent" give the minimum and maximum across sections.

• Every text clearly avoids repeating the same word. In all seven, identical neighbours occur at 0.12–0.88 times chance — in recipe books and in treatises, in Tuscan and in eastern Lombard alike. The Voynich (1.34) is the opposite.
• One-glyph neighbours sometimes exceed chance, but because of set phrases. In the Bergamo surgery (1.24), the main cause is the recurring “molti volti” (“many times”), which forms a one-glyph pair in northern spelling. In the Tuscan surgeries (1.04 and 1.08) it is preposition pairs such as “della / delle” and “dela / del”; in the Catalan sections of the Ravenna book (1.03) mainly “per fer” (“in order to”).
• The closest text is a late 13th-century Florentine recipe book. The New Haven recipes reach 0.87, the highest value among the medieval recipe, medical and herbal texts measured so far (the previous highest were the Venetian recipe book and a 14th-century cookbook at about 0.6). It is still below chance, and the text is short (z -1.9). Texts of other kinds can be higher, such as Pico della Mirandola in Chapter 8 (1.13), but there only identical words are enriched, while one-glyph neighbours are avoided.
• The same holds for a text from the Voynich's own time and region. The Bergamo surgery is a mid-15th-century northern Italian manuscript, overlapping the vellum date (1404–1438). Its adjacent similarity is 0.41, clear avoidance.
• Glyph predictability stays that of ordinary language. h2 is 3.0–3.3 bits in all of them, far from the Voynich's 2.24.

Caveats

• Critical texts. Expansion of abbreviations, word division and punctuation follow the editors' decisions, so the spelling differs somewhat from the manuscripts themselves.
• Mixed languages. The Ravenna recipe book alternates Latin and vernacular, and the Almansore and the surgeries contain Latin phrases. They were measured as they are.
• Length. The New Haven recipes (about 7,600 words) and Ravenna section B (about 4,300) are short, so their values vary more. The Almansore ranges from 0.38 to 0.68 across sections, but every section is well below 1.

In short, adding seven northern Italian and Tuscan recipe books, surgeries and medical treatises still yields no medieval text in which both identical and one-glyph-different words cluster next to each other. Identical repeats, in particular, are clearly avoided in all of them. This property of the Voynich cannot be explained by its genre (recipe or medical text) or by northern Italian origin.

Sources: I. Zamuner – E. Ruzza, I ricettari del codice 52 della Historical Medical Library di New Haven (XIII sec. u.q.), Firenze, Olschki, 2017 / C. Lemme, Il ricettario del ms. 215 della Biblioteca Classense di Ravenna (ff. 93r-156v), Alessandria, Edizioni dell'Orso, 2022 / E. Ventura, La «Chirurgia Magna» di Bruno da Longobucco in volgare, Berlin/Boston, De Gruyter, 2020 / R. Piro, L'Almansore. Volgarizzamento fiorentino del XIV secolo, Firenze, SISMEL, 2011 / E. Artale – M. Panichella, “Un volgarizzamento toscano della Chirurgia di Ruggero Frugardo”, Bollettino dell'Opera del Vocabolario Italiano XV (2010), 227–98 / I. Zamuner, “Il volgarizzamento toscano della Chirurgia di Ruggero Frugardo nel codice 2163 della Biblioteca Riccardiana”, BOVI XVII (2012), 245–332. We thank Prof. Ilaria Zamuner for providing the editions and agreeing to the publication of the results.

42. A fourth night: searching with the pooled yardstick

Chapter 40 showed that classifiers shown too few fake pages make the model look harder to distinguish than it is. For the fourth overnight run we therefore switched the yardstick to the “pooled classifier”. It is trained on all fake pages from the six evaluation runs together, about six times as many fake pages as manuscript pages.

• Explorer. A single Opus 5.5 instance (reasoning effort “high”), running for about 19 hours from 11:35 on 24 October to 6:30 on 25 October. The starting point was the Chapter 39 final setting.
• Aim. To match the “direction” of page-to-page variation to the manuscript (the Chapter 39 diagnosis).
• Outcome in brief. 110 candidates were evaluated and 2 adopted. The result was graded with a final exam shown to no search (version 3: the eight classifiers of Chapter 40).

Final exam (version 3)

Distinguishability (AUC) in the final exam, mean of six seeds. 0.5 means indistinguishable. * = classifiers never used in any search. Bold: values that moved by 0.02 or more. Pooled values vary by about 0.01–0.02 per seed.

• Only the linear classifier (regression) moved towards 0.5. The pooled regression fell from 0.56 to 0.54. This agrees with the explorer's own re-measurement on seeds not used for selection (0.562 → 0.526).
• The tree classifiers did not move. Pooled gradient boosting gives 0.68 and 0.69, the pooled random forest 0.68 and 0.68; the differences are within noise. The explorer itself reported that gradient boosting stayed around 0.70 and that the goal was not reached.
• All key statistics stayed within their target ranges. Once-only words, identical neighbours and glyph predictability moved slightly closer to the targets.

From the explorer's report. “Search” = search seeds (101–106); “Unused” = seeds never used for selection (113–118). Values on the search seeds (101–106) are optimistic because the final was chosen among many candidates (winner's curse). The unbiased comparison is the “unused seeds” (113–118) columns.

What was learned

• Seed-to-seed noise is larger than the effect of the settings. With the same setting, changing the set of seeds moves O by 0.015–0.03. A candidate that scored 0.650 for gradient boosting on the search seeds scored 0.712 on other seeds. Comparing candidates one at a time cannot detect improvements smaller than about 0.03.
• Model pages are too predictable from nearby pages. We regressed each page on the mean of its two nearby pages, direction by direction along the manuscript's principal components. The manuscript inherits strongly from nearby pages along the main directions (slope 0.5–0.7) and hardly at all along minor ones (0.1–0.3). The model inherits uniformly, at about 0.6 in every direction, and its page-specific “novelty” (the part not predictable from nearby pages) is only 0.6–0.8 times the manuscript's. The Chapter 39 diagnosis (averaging neighbours dilutes variation) pointed the right way, but the remedy was the opposite. Strengthening dependence along the main directions made things worse: what is missing is not dependence but novelty.
• The strong classifiers' cues are spread over many features. Classifiers using one feature at a time ranked the share of once-only words, mean word length, word-initial glyphs and type/token ratio highest. In each, the mean matches the manuscript, but the page-to-page spread (standard deviation) is about 0.8 times the manuscript's. Fixing one of them only made the classifier switch to another cue (two- and three-glyph sequences), and the overall value hardly moved.
• Experimental designs pulled effects out of the noise. Midway, the explorer switched to 16-run orthogonal designs that move 15 settings up and down at the same time. This gave clear estimates of some effects. For example, lowering the copying rate (p_cite) makes pages harder to distinguish, in the same direction in two designs (differences 0.052 and 0.029, standard error about 0.007). The final setting was built from these estimates.
• None of the new mechanisms helped. The explorer added seven kinds of mechanism, such as splitting dependence on nearby pages by direction, adding random page-specific novelty, and letting vocabulary richness vary by page. None beat the starting point in the search evaluation, and none is in the final setting (with default values they generate exactly as before).

Caveats

• Winner's curse. The final was selected from 110 candidates, and its first value on the search seeds (O 0.615) includes luck. Judge by the final exam and the unused seeds.
• What improved. The improvement this time is only in the linear classifier and in the penalty for key statistics. Since the yardstick was strengthened, the tree classifiers have stayed at 0.68–0.69.

In short, the fourth search could not lower the tree classifiers by tuning settings. Instead, it became clear why they do not move: model pages resemble their neighbours too closely and lack page-specific novelty, and strong classifiers see this difference spread across many features. What is needed next is not more tuning but a mechanism that gives each page manuscript-like novelty, and a way of comparing candidates that is not swamped by seed noise.

Worth trying next

• Giving each page its own novelty: Chapter 42 showed that model pages are too predictable from nearby pages, with only 0.6–0.8 times the manuscript's page-specific novelty. The manuscript inherits from nearby pages along its main directions and hardly at all along minor ones. Try a mechanism that matches this pattern of inheritance and novelty across all the features the classifiers use (two- and three-glyph sequences, words, repetition).
• Comparisons that are not swamped by seed noise: changing a setting slightly currently produces entirely different fake pages, so seed noise (0.015–0.03 in O) sits on top of every difference between candidates. Separating the random stream for each page and comparing candidates on the same random numbers (common random numbers) would make small improvements visible. Combine this with orthogonal designs (Chapter 42).
• Short pages: pages with fewer words (under 80) are easier to distinguish. Study separately how the short texts of the star and jar sections are written.
• More Italian medical texts: Chapter 41 measured seven critical editions provided by a specialist. When the ReMediA site (OVI's corpus of medieval medical texts) is back online, check further texts such as 15th-century northern Italian herbals, within what its search interface allows.
• Combining sheet order with other evidence: word preferences alone cannot fix the order of sheets (Chapter 15), so combine them with ink offsets, handwriting and continuity of drawings — in particular, check where the f49/f56 sheet belonged.

Methods and limitations

• ch, sh, cth, ckh, cph, cfh were counted as single glyphs; h2 values depend on this convention.
• Words containing unreadable characters (? or *) were excluded (about 0.7% of the total).
• The natural languages compared in Chapter 4 are one work each in English (Moby-Dick) and Latin (Caesar, De bello Gallico). Chapters 6–8 widened this to Bibles in 93 languages, Latin recipe collections and herbals, and contemporary texts. The contemporary vernacular recipe collections are two: Middle English (ed. Henslow) and Italian (ed. Zambrini).
• The chance baseline is always “shuffle words within the same page”. Page-level vocabulary bias is included in the baseline, so differences reflect structure finer than the page.
• p-values are from permutation tests (comparison with random reshuffles) and cannot be finer than the number of repetitions. The closeness of similar words was estimated at z ≈ 21 from 500 shuffles.
• From Chapter 20 on, model comparisons are means of six generations with different random seeds. In the autonomous loop of Chapter 24 all candidates were evaluated on the same six seeds and finally checked on six other seeds; because comparing many candidates on the same seeds selects settings that happen to suit them, the fresh-seed values are the final evaluation. Python’s set ordering (PYTHONHASHSEED) was not fixed in the autonomous loop of Chapter 24; it was later found to make results fluctuate as well (Chapter 31), and Chapter 31 averages over three orderings.