P001

Part 1: Introduction Text

Hi, X. My name is Y, and I'm going to be walking you through this session.

You probably already know, but let me explain why we've asked you to come here today. We're testing an electricity display that we're working on so we can see what it's like for actual people to use it.

I want to make it clear right away that we're testing the display, not you. You can't do anything wrong here. In fact, this is probably the one place today where you don't have to worry about making mistakes.

We want to hear exactly what you think, so please don't worry that you're going to hurt our feelings. We want to improve it, so we need to know honestly what you think.

As we go along, I'm going to ask you to think out loud to tell me what's going through your mind. This will help us.

If you have questions, just ask. I may not be able to answer them right away, since we're interested in how people do when they don't have someone sitting next to them, but I will try to answer any questions you still have when we're done.

We have a lot to do, and I'm going to try to keep us moving, but we'll try to make sure that it's fun, too.

Preliminary Questions

Before we look at the device, I'd like to ask you just a few quick questions. First, what's your occupation?

Roughly what do you know about your electricity use?

Do you know your bill, kWh?

How do you feel about learning about your electricity use?

Part 2: Looking at the Device and Unaided Exploration (i.e., see what the user can do by themselves)

First, I'm just going to ask you to look at this device and tell me what you think it is, what strikes you about it, and what you think you would do first.

For now, don't actually do anything. Just tell me what you would do.

And again, as much as possible, it will help me if you can try to think out loud so we know what you're thinking about.

Let them interact and think out loud

--When done: If you had to take a guess, what do you think it is about?

Ok. Now, if you were at home, what would you do first?

...continue

Notes (just let them explore, but take notes on these items):

1. How much can the person do with no instruction; just “use the device”?
2. Without any instruction, what do they think the purpose of the device is, the features are, the buttons do?
3. Can the person tell the state of the device just by looking at it?
4. Can the person tell the action alternatives just by looking at the device?
5. What is the mental model of the participant? Does this match the designer's model and the system image?
6. What are the errors people make? How do we fix them? Are they detectable, minimal consequences, and reversible?
7. What are the mappings between controls and functions?
8. What is the gulf of execution? What is the difference between intentions and allowable actions?
9. What is the gulf of evaluation? What is the amount of effort required to interpret the physical state of the system and determine whether intentions have been met?
10. Can the person determine the functions of the device unaided?
11. Can the person tell what actions are possible unaided?
12. Can the person determine the mapping between intentions and physical movement unaided?
13. Can the person perform intended actions unaided?
14. Can the person tell if the system is in the desired state unaided?
15. Can the person determine mapping from system state to interpretation unaided?
16. Can the person tell what state the system is in unaided?
17. Can the person determine the relationships between actions and results?
18. Can the person determine the relationship between the controls and their effects?
19. Can the person determine the relationship between the system state and what is visible?
20. Can we adjust the arrangement so as to reduce the number of labels while improving c comprehension (i.e., can we find natural mappings)?
21. Can people muddle through without reading the instructions or help from the experimenter?

Part 3: Goal-directed tests

1. Can you think of something you'd like to learn from the device? If you wanted to learn that, what would you do?
2. Can you access the calendar?
3. Can you show a bar graph of monthly consumption?
4. What is the monthly bill?
5. How much electricity is being used right now?
6. What is the current price of electricity?
7. Can you switch from calendar to bar graphs and back?
8. How many kWh were used yesterday?
9. How many kWh are projected to be used tomorrow, next week, month...
10. Can you switch from bar graphs to pictures?
11. Can you switch from pictures to bar graphs?
12. Can you switch from calendar to pictures?
13. Can you switch from pictures to calendar?
14. Can you switch from bar graphs to calendar?
15. Can you switch from calendar to bar graphs

Part 4: Trunk test

Have participant close eyes and take them to randomly selected pages (actually use a random number generator for this). Can they answer the following questions?

1. What is this?
2. What page am I on? (page name)
3. What are the major sections of this site? (sections)
4. What are my options at this level? (local navigation)
5. Where am I in the scheme of things? (“you are here indicator”)
6. How can I get back to pictures and the default page?

Part 5: In the dark test

Turn off the lights. Secretly give them the control upside-down. Can they still use the device?

Part 6: Dual task

Can participants dual task (e.g., n-back) and also use the IHD?

Part 7: Destructive Testing

Can they break it? Not physically.

Part 8: Open-ended suggestion period

Just discuss it open-ended with them. Take suggestions.

Navigation

• Go back to Technology Usability Testing Notebook 2012/06/26
• Notes for Technology Usability Testing (IHD_field_trial)
• Go back to IHD_field_trial