Show HN: IR_evaluation – Information retrieval evaluation metrics in pure Python

[plurch]

I created this library for personal use and also to solidify my knowledge of information retrieval evaluation metrics. I felt that many other libraries out there are overly complex and hard to understand.

These metrics are useful in many different domains such as search engines, recommender systems, and RAG with LLMs.

This implementation has easy to follow source code and unit tests. Let me know what you think and if you have any suggestions, thanks for checking it out!

[jonathan-adly]

Great work! Honestly it helps so much just explaining these metrics for folks.

Early on RAG was an art, now when things are stabilized a bit, it’s more of a science - and vendors should at a minimum have some benchmarks.

[plurch]

Thanks! Yes, evaluations and benchmarks are fundamentally important. It's the only way to know if you are actually making improvements.