[accraze]

Three papers stick out for me in the IML / participatory machine learning space this year:

1) Michael, C. J., Acklin, D., & Scheuerman, J. (2020). On interactive machine learning and the potential of cognitive feedback. ArXiv:2003.10365 [Cs]. http://arxiv.org/abs/2003.10365

2) Denton, E., Hanna, A., Amironesei, R., Smart, A., Nicole, H., & Scheuerman, M. K. (2020). Bringing the people back in: Contesting benchmark machine learning datasets. ArXiv:2007.07399 [Cs]. http://arxiv.org/abs/2007.07399

3) Jo, E. S., & Gebru, T. (2020). Lessons from archives: Strategies for collecting sociocultural data in machine learning. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 306–316. https://doi.org/10.1145/3351095.3372829

Also a great read related to IML tooling for audio recognition:

1) Ishibashi, T., Nakao, Y., & Sugano, Y. (2020). Investigating audio data visualization for interactive sound recognition. Proceedings of the 25th International Conference on Intelligent User Interfaces, 67–77. https://doi.org/10.1145/3377325.3377483

[angrais]

These do seem interesting, thanks for sharing.

Also, what do you mean by "participatory" in the context of machine learning? Is there a seminal paper that defines it?

I ask as in HCI and other fields, participatory had a VERY defined meaning that in short, I'd about equal power, democracy, and inclusivity. I can't understand how that applies to ML and would like to learn more, hence asking you.

[accraze]

I think "participatory" means something similar here within an ML context. It favors building community-based algorithmic systems and focuses on lowering the barrier to participation, so that non-expert users can be involved during the machine learning development cycle.

I'm not aware of any seminal papers per-say, although here are a few that I've read recently... first one is something I maintain at $DAYJOB:

1) Halfaker, A., & Geiger, R. S. (2020). Ores: Lowering barriers with participatory machine learning in Wikipedia. ArXiv:1909.05189 [Cs]. http://arxiv.org/abs/1909.05189

2) Martin Jr. , D., Prabhakaran, V., Kuhlberg, J., Smart, A., & Isaac, W. S. (2020). Participatory problem formulation for fairer machine learning through community based system dynamics. ArXiv:2005.07572 [Cs, Stat]. http://arxiv.org/abs/2005.07572

Also checkout PAIR: https://research.google/teams/brain/pair/

[gradys]

Thank you! We have very similar interests! Especially the first one and the interactive sound recognition one. Any other work in IML you'd recommend?

[accraze]

Alot of IML seems to focus on building interfaces, so this one was pretty good:

1) Dudley, J. J., & Kristensson, P. O. (2018). A review of user interface design for interactive machine learning. ACM Transactions on Interactive Intelligent Systems, 8(2), 1–37. https://doi.org/10.1145/3185517