I'd probably start by designing a perfect player. It has a dictionary, and it can narrow it down to every possible word according to the data it knows. It always guesses so as to gain the most information or eliminate the most possible words.

Then I could run the ideal player against every word there is to see how it scores—how fast it guesses the word. That's a baseline.

Each less than ideal opponent, I could implement and then run a similar test. This way I can rank the algorithms. I can see which modifications have an effect, which don't, and how much.

I'd implement the lesser guessers by making them some degree of forgetful. It has a smaller vocabulary. It forgets some of the facts it has learned in the course of the game. By losing some of its rules, it would make worse guesses.

I'm imagining some data structure that has all the known game information in it, and a routine that has a dictionary and presents all words that match the data. The lesser guesser would randomly lose parts of the real data structure before passing to the dictionary selector.