Phase 3 · Evaluation Design
For our evaluation we decided to do an empirical evaluation. The subjects of our evaluation will be a combination of friends and family who are willing to help out, and have backgrounds that are representative of our target users. Since iCoach is designed to be used by everyone from intramural football to college or the NFL we recruited both high school coaches, intramural team members, and unexperienced coaches alike.
For the evaluation evaluators will explain to the participant the function of the application and will give the participant 5 min to play with the application and to read any help files or tutorials that they feel they need to. Once the user has completed this initial trial period of the app we ask them to complete a simple series of tasks, in which they create a new play, attach a video to the play, and deploy the play to a selected group of players. We then have the user fill out a post-testing questionnaire in which they answer some simple questions about their experience.
The data for the study will consist of two parts. First, the amount of time the user takes to accomplish each task will be measured. Evaluators will observe the user and count the number of clicks and time each task. For the second part, of the evaluation users will be asked to complete a post-test survey about their experience, and the results will be collected using survey.vt.edu. These results will help us gauge user comfort and overall experience.
Data will be collected using a combination of direct observation and screen capture sofware, and data will be analized using exel.