[drw]

Last month we added 2800, the vast majority automatically derived, though we also prize expert review.

We are thinking of doubling that rate next month.

I'm not sure much can be done about the predominance of certain puzzle types in the very early Rush puzzles. Would you prefer more hung pieces?

But in the later levels, mixing it up by themes/tags is an interesting idea, about which I will make some measurements.

[noxvilleza]

Yeah overall some theme diversity. Would also be interested to know the median Elo rating by tag - since that'd serve a good sort of table-of-contents of order for people to learn chess tactics.

What is also pretty interesting would be a (fuzzed) per-tag Elo rating on the user - so they can see which themes you are good/bad at relatively (for example, if you constantly get problems on Trapped Pieces wrong your rating on those problems would be lower than your average Puzzle Rating).

[drw]

Yes, the rating differs by theme quite a bit.

  +-------------------+------------+
  | theme/tag         | mdn_rating |
  |-------------------+------------|
  | Opposition        | 1725       |
  | Passed Pawns      | 1494       |
  | Rooks on Seventh  | 1408       |
  | Pawn Endgame      | 1353       |
  | ...               | ...        |
  | Smothered Mate    | 549        |
  | Stalemate         | 533        |
  | Mate in 2         | 436        |
  | Mate in 1         | 399        |
  +-------------------+------------+

I also like the idea of showing the user's +/- for each theme. Though the data for some themes is better than data for others.

[SamBam]

Why such a large number? Do players really go through thousands of puzzles a month, and remember them all?

If you produced fewer, would that increase the proportion with expert review?

[Nemerie]

Casper Schoppen broke all the previous records in puzzle rush (a mode where you get to solve as much puzzles as possible in 5 minutes) by memorizing most of high rated puzzles. After that chess.com decided to add a lot of new ones.

https://www.chess.com/article/view/puzzle-rush-confirmed-rec... https://www.chess.com/news/view/new-puzzle-rush-chess

Although I don't think there were lots of players who tried to use this tactic among all the ones who occasionally train on chess.com.