[nwiswell]

> It seems clear to me that there is an optimal starting word

I took one step further and calculated the optimal starting word for obtaining green matches (I assume that this also makes it likely to produce yellow matches, although I did not explicitly optimize for that).

Beginning with the full list of 5-letter words, I calculated the frequency of each letter of the alphabet in each of the 5 possible positions for a 5 letter word.

Then I iterated through the list a second time, this time assigning a score for each word equal to the sum of frequencies for each letter in its respective position.

By a significant margin, the highest score is SLATE (over 1400). Runners up (over 1300) are SAUTE, SHIRE, and CRATE.

Caveat: this approach assumes that all possible words are equally likely to be the answer.

[codefined]

Interestingly, doing a similar analysis where we bucketed each word into 243 (3^5) buckets based on the possible result, we found "RAISE" as the best word. Source[0]

[0] https://gist.github.com/popey456963/a654e98d0180566b897b70ee...

[nwiswell]

RAISE also sounds pretty plausible.

How is the score calculated? What is the trade-off between expectation greens / expectation yellows?

Intuitively I expect that greens are significantly more "valuable" than yellows, since they reduce the search space much more...

Maybe this is all the wrong approach and the right metric is how many legal answers remain after the first guess?

[PennRobotics]

I would say that the distribution of yellow letters in the remaining word determines the value. If you have a yellow "X" or "Q" or one yellow vowel after testing all 5 vowels, that is a lot more meaningful than a yellow "T", which could really go anywhere. A yellow "G" but grey "I" tells me (at least intuitively, not quantitatively) a bit more than a green "I" and grey "G".

A recent word had a single vowel. This made it reasonably easy to guess where those vowels could go and to also choose a consonant-heavy next word that would really tear apart the search space if even one consonant was valid.

I think I'll switch to using RAISE or SLATE as a first choice. It was ADIEU before, but there's almost never an A in slot 1, U in slot 5, or D in slot 2, and I don't feel like any of these letters has a significantly most common position with just the info from these 5 letters. The yellow/grey letters are helpful after ADIEU, but I still need to explore at least 4 more letters (including O, and usually N or C) to even get a good idea of where any matches might go. In the moments that letter 4 is "E" while guessing ADIEU on round 1, that actually doesn't help me much with round 2. In fact, then I'm faced with the decision to explore unguessed letters in general, words with two "E"s, and/or words with "E" in 4th position. (same with "I" in 3rd)

[EveYoung]

I've implemented your last suggestion of scoring words by how much they reduce the pool of potential candidates (12k+). Testing it against the 1,000 most frequent words (targets), the word LORES seems to work best on average.

[theshrike79]

I did the same and ended up with ARISE.

My word list was alphabetical and the system picked up the first one :P

[Nursie]

Interesting. I've been using ORATE as it knocks out a lot of the top frequency letters from the english language as a whole. But I guess the distribution in five letter words, and more so in a subset of five letter words, may skew differently.

[bee_rider]

From what I saw, T shifts down noticeably when restricted to 5 letters exactly. Intuitively, it sort of make sense -- there are a ton of short T words. ('that' and friends).

[paulcole]

> Beginning with the full list of 5-letter words, I calculated the frequency of each letter of the alphabet in each of the 5 possible positions for a 5 letter word.

There’s a great many 5-letter words that the creator will never consider because they’re too obscure. Treating all 5-letter words as possible is a mistake when calculating strategy for this.

High-frequency letters will be over-represented in your analysis.

[CSSer]

GP might’ve pulled the wordlist from the site. It’s in the bundle iirc.

[nwiswell]

Yep, it's the site wordlist.

[alkonaut]

Nice. I tried a simpler method of only looking at the wikipedia table of overall letter frequency in dictionaries (not in 5 letter words) and the vowels are high, especially E and I followed by A. Then I checked the frequeny of starting letter and that is led by "S". So my guess for a 5 letter word became one with at least two vowels from E,I,A, starts with an S, and has the remaining two characters from the reasonably high frequency set (R, T, N, etc). My guess then was SIREN which has that property. Just out of curiosity, if you still have the output from your program, how bad was SIREN compared to SLATE?

[tcskeptic]

That is a really interesting approach -- if you remove the words with any of the letters in SLATE, what is the highest ranked word? (If you have it to hand)