University of Gothenburg Webinars on Citizen Science: Heteromation in Citizen Science: The Division of Labor Between Citizens, Experts, and Machines – Presenters: Marisa Ponti & Anna Jia Gander. Gothenburg University, 5th of February 2021, 10:00-12:00 CET. Zoom-link: https://gu-se.zoom.us/j/6218196596
Citizen science is a promising field for the creation of human-machine systems with increasing computational abilities, as several projects generate large datasets that can be used as training materials for machine learning models. We present the results of a recent literature review aimed to identify the forms of human-machine learning integration in citizen science projects. The fifty articles examined in this systematic review report on projects combining human and machine efforts for analyzing, coding, classifying, and clustering data provided, for example, by cameras and telescope images. Machine learning is used at various stages of the data life cycle, through algorithms that perform tasks like classification, regression, clustering, and association. The findings highlight the character of the projects as heteromated systems, wherein human participation still remains crucial, and volunteer and ML efforts are often positioned as complementary rather than mutually exclusive. While leveraging the complementarity of strengths is one of the main arguments to combine humans and machines and enhance their respective capabilities, essentializing the attributes of humans and machines should be avoided. Treating these attributes as stable and natural does not take into account that cognitive work will be shifting between humans and machines, as the list of research tasks that machines can do is growing, although algorithms are still second to humans on recognizing patterns and they have longer learning curves. The findings can help researchers and practitioners to better understand human-machine integration in citizen science and point to unexplored areas.
TEKNO virtual seminar: The citizen-in-the-loop in citizen science classification projects – Presenters: Anna Jia Gander and Marisa Ponti. October 20, 2020. https://doi.org/10.5281/zenodo.4109066
Function allocation refers to deciding which tasks should be allocated to humans and which ones to machines. This allocation is important in citizen science projects to operationalize the concept of complementarity and develop systems wherein both humans and machines can benefit from collaboration. Machine learning has been used in several projects to improve classification tasks, such as plants, animals, and galaxies, among others. We present the findings of a descriptive study based on document analysis that aims to examine function allocation between citizen scientists, experts, and machines in 12 citizen science classification projects. We posit this work as a first step towards understanding function allocation and configurations of citizen-in-the-loop. The results indicate that experts are involved in every aspect of the loop, from annotating or labelling data to giving them to algorithms to train and make decisions from such predictions. Experts also test and validate models to improve accuracy by scoring their outputs, when algorithms are not able to make the right decisions. While experts are the humans mainly involved in the loop, citizens are involved as well at various stages of the process. We present two examples of citizens-in-the-loop: when algorithms provide incorrect suggestions and when algorithms are active learners.