Touchstone2: An Interactive Environment for Exploring Trade-offs in HCI Experiment Design

Alexander Eiselmayer (1) , Chat Wacharamanotham (1) , Michel Beaudouin-Lafon (2) , Wendy Mackay (2)

1 UZH - Universität Zürich [Zürich] = University of Zurich
2 EX-SITU - Extreme Interaction

Touchstone2offers a direct-manipulation interface for gener-ating and examining trade-offs in experiment designs. Basedon interviews with experienced researchers, we developedan interactive environment for manipulating experimentdesign parameters, revealing patterns in trial tables, and esti-mating and comparing statistical power. We also developedTSL, a declarative language that precisely represents experi-ment designs. In two studies, experienced HCI researcherssuccessfully usedTouchstone2to evaluate design trade-offsand calculate how many participants are required for par-ticular effect sizes. We discussTouchstone2’s benefits andlimitations, as well as directions for future research.

Mots clés: Reproducibility; Poweranalysis; Counterbalancing; Randomization; Experiment design

Domaines: Interface homme-machine [cs.HC]

Alexander Eiselmayer, Chat Wacharamanotham, Michel Beaudouin-Lafon, Wendy Mackay. Touchstone2: An Interactive Environment for Exploring Trade-offs in HCI Experiment Design. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, May 2019, Glasgow, United Kingdom. pp.1--11, ⟨10.1145/3290605.3300447⟩. ⟨hal-02127273⟩