[Retric]

Also, if your using the DB as a queue index on the status element. Inserts are still low overhead and the DB can handle poling that table 10,000 time a second without issue.

Edit: Just don't pole the 'finished' status as that can have a lot of elements in it.

[koenigdavidmj]

A lot of databases can also provide partial indices, which are only maintained and searched when a WHERE clause matches (example: in this case, WHERE status != 'filtered' would be a good choice).