Measuring Uncertainty in Assessments through Mouse Tracking

by

Print Friendly, PDF & Email

PI: Dr. Ross Larsen
Co-PI’s: Dr. Jeff Jenkins and Dr. Charles Graham

Evaluation of how well the academic objectives of the proposal were met

Our project sought to use mouse tracking to develop a measure of uncertainty and validate it with self-report with self-report measures of uncertainty as seen in Figure 1.

Several developments during data collection caused some of our methods to change. For example, we ended up using Multiple Regression rather than Confirmatory Factor Analysis to understand the relationships between the mouse tracking metrics and self-reported uncertainty. Nevertheless, we had success in making a predictive model and have presented our findings at the Americas Conference on Information Systems (Jenkins, Larsen, Bodily, Sandberg, Williams, Stokes, Harris, & Valaicich, 2015). The full paper can be seen in Appendix A. We have not spent all of our research funds so we are taking the next steps in researching mouse tracking as described below.

Evaluation of the mentoring environment

This project has been an excellent experience in mentoring students in their professional aspirations. We had weekly meetings and trainings where (a) students learned how to use predictive modeling, (b) use existing theoretical models to inform their work, and (c) have a greater understanding of the programming languages necessary to do mouse tracking. The students have and will continue to develop publications that will benefit them.

List of students who participated and what academic deliverables they have produced or it is anticipated they will produce

*Bodily R., Jenkins, J. L., Larsen, R. A. A., *Sandberg, D., *Williams, P., & Valacich, J. S. (In preparation) Predicting response certainty using mouse-tracking statistics. Journal of the Association for Information Systems.

    • This study will be co-authored with Robert Bodily, Daniel Sandberg, and Parker Williams. Robert is a graduate student here at BYU while Daniel was an undergraduate and Parker continues to be an undergraduate student at BYU. This study looked at using the mouse to track uncertainty of a subject.

*Hedges, A., Larsen, R. A. A., Jenkins, J., *Williams, P., & Sandberg, D. (In preparation) Context and learning: How mouse tracking correlates with mnemonic discrimination. Hippocampus

    • This paper will be co-authored with Ariana Hedges, and Parker Williams. Ariana is a graduate student here at BYU while Parker is an undergraduate student here at BYU. This study looks at how mouse tracking and fMRI can be used to track hippocampus activity when uncertainty is present.

Bodily, T., Jenkins, J., Larsen, R. A. A., Williams, P., & Sandberg D. (In preparation) Survey response fatigue: Mouse tracking and fMRI results TBD

    • This paper will be co-authored with Ty Bodily and Parker Williams. Ty is a graduate student here at BYU, Parker is an undergraduate student here at BYU. This study looks at whether survey fatigue can be detected by mouse tracking and whether that fatigue can also be seen with an fMRI.

Jenkins, J. L., Larsen, R. A. A., *Bodily, R., *Sandberg, D., *Williams, P., *Stokes, S., *Harris, & S., Valacich, J. L. (2015). A Multi-Experimental examination of analyzing mouse cursor trajectories to gauge subject uncertainty. Paper presented at the Americas Conference on Information Systems (AMCIS), Fajardo, Puerto Rico. – Runner up for best paper award

  • This presentation was co-authored with Robert Bodily, Daniel Sandberg, Parker Williams, Steve Stokes, and Scott Harris. Robert Bodily and Steve Stokes are graduate students here at BYU and Daniel Sandberg, Parker Williams and Scott Harris are undergraduate students here at BYU. This study was a result of a MEG grant and studied whether mouse tracking can be used as a proxy for uncertainty.

Description of the results/findings of the project

Briefly, mouse tracking can indeed predict self-reported uncertainty of a subject. Mouse tracking has the greatest success in predicting uncertainty when a person is moderately uncertain about the answer rather than high or low uncertainty. Full details are found in the article in Appendix A. Work is continuing in using fMRI and mouse tracking in concert to understanding the workings of the hippocampus and survey fatigue.

Description of how the budget was spent

As noted previously, our funds were not all spent in the two year window and we continue the work. The majority of the money we spent was funding graduate and undergraduate students. A small amount was used in paying users of mechanical turk to take surveys and in traveling to conferences.