P004

P004

Preliminary Questions

Before we look at the device, I'd like to ask you just a few quick questions. First, what's your occupation?

• Grad student

Roughly what do you know about your electricity use?

• Don't know much, just came to the states

Do you know your bill, kWh?

• $14
• Don't know kWh

How do you feel about learning about your electricity use?

• Great, you can learn how to cut costs

Part 2: Looking at the Device and Unaided Exploration (i.e., see what the user can do by themselves)

First, I'm just going to ask you to look at this device and tell me what you think it is, what strikes you about it, and what you think you would do first.

For now, don't actually do anything. Just tell me what you would do.

And again, as much as possible, it will help me if you can try to think out loud so we know what you're thinking about.

Let them interact and think out loud

--When done: If you had to take a guess, what do you think it is about?

Ok. Now, if you were at home, what would you do first?

...continue

Notes (just let them explore, but take notes on these items):

1. How much can the person do with no instruction; just “use the device”?
2. Without any instruction, what do they think the purpose of the device is, the features are, the buttons do?
3. Can the person tell the state of the device just by looking at it?
4. Can the person tell the action alternatives just by looking at the device?
5. What is the mental model of the participant? Does this match the designer's model and the system image?
6. What are the errors people make? How do we fix them? Are they detectable, minimal consequences, and reversible?
7. What are the mappings between controls and functions?
8. What is the gulf of execution? What is the difference between intentions and allowable actions?
9. What is the gulf of evaluation? What is the amount of effort required to interpret the physical state of the system and determine whether intentions have been met?
10. Can the person determine the functions of the device unaided?
11. Can the person tell what actions are possible unaided?
12. Can the person determine the mapping between intentions and physical movement unaided?
13. Can the person perform intended actions unaided?
14. Can the person tell if the system is in the desired state unaided?
15. Can the person determine mapping from system state to interpretation unaided?
16. Can the person tell what state the system is in unaided?
17. Can the person determine the relationships between actions and results?
18. Can the person determine the relationship between the controls and their effects?
19. Can the person determine the relationship between the system state and what is visible?
20. Can we adjust the arrangement so as to reduce the number of labels while improving c comprehension (i.e., can we find natural mappings)?
21. Can people muddle through without reading the instructions or help from the experimenter?

Part 3: Goal-directed tests

1. Can you think of something you'd like to learn from the device? If you wanted to learn that, what would you do?
2. Can you access the calendar?
3. Can you show a bar graph of monthly consumption?
4. What is the monthly bill?
5. How much electricity is being used right now?
6. What is the current price of electricity?
7. Can you switch from calendar to bar graphs and back?
8. How many kWh were used yesterday?
9. How many kWh are projected to be used tomorrow, next week, month...
10. Can you switch from bar graphs to pictures?
11. Can you switch from pictures to bar graphs?
12. Can you switch from calendar to pictures?
13. Can you switch from pictures to calendar?
14. Can you switch from bar graphs to calendar?
15. Can you switch from calendar to bar graphs

Part 4: Trunk test

Have participant close eyes and take them to randomly selected pages (actually use a random number generator for this). Can they answer the following questions?

1. What is this?
2. What page am I on? (page name)
3. What are the major sections of this site? (sections)
4. What are my options at this level? (local navigation)
5. Where am I in the scheme of things? (“you are here indicator”)
6. How can I get back to pictures and the default page?

Part 5: In the dark test

Turn off the lights. Secretly give them the control upside-down. Can they still use the device?

Part 6: Dual task

Can participants dual task (e.g., n-back) and also use the IHD?

Part 7: Destructive Testing

Can they break it? Not physically.

Part 8: Open-ended suggestion period

Just discuss it open-ended with them. Take suggestions.

Navigation

• Go back to 2012/07/06
• Notes for Technology Usability Testing (IHD_field_trial)
• Go back to Technology Usability Testing
• Go back to IHD_field_trial