P005

P005

Preliminary Questions

Before we look at the device, I'd like to ask you just a few quick questions. First, what's your occupation?

• Graduate student

Roughly what do you know about your electricity use?

• You pay your bill every month
• Higher in the summer

Do you know your bill, kWh?

• $30-$55 in the summer
• Don't know kWh

How do you feel about learning about your electricity use?

• Would like to learn more
• Terminology and language are barriers to learning now
• Graphs on the bill don't say much

Part 2: Looking at the Device and Unaided Exploration (i.e., see what the user can do by themselves)

First, I'm just going to ask you to look at this device and tell me what you think it is, what strikes you about it, and what you think you would do first.

For now, don't actually do anything. Just tell me what you would do.

And again, as much as possible, it will help me if you can try to think out loud so we know what you're thinking about.

Let them interact and think out loud

--When done: If you had to take a guess, what do you think it is about?

Ok. Now, if you were at home, what would you do first?

...continue

Notes (just let them explore, but take notes on these items):

1. How much can the person do with no instruction; just “use the device”?

• Understood what each slide was trying to illustrate

1. Without any instruction, what do they think the purpose of the device is, the features are, the buttons do?

• Was a bit confused about the remote "Weeks (up arrow)" and "Hours (down arrow)"

1. Can the person tell the state of the device just by looking at it?
2. Can the person tell the action alternatives just by looking at the device?
3. What is the mental model of the participant? Does this match the designer's model and the system image?

• Yes

1. What are the errors people make? How do we fix them? Are they detectable, minimal consequences, and reversible?
2. What are the mappings between controls and functions?

• Probably need an instruction manual

1. What is the gulf of execution? What is the difference between intentions and allowable actions?
2. What is the gulf of evaluation? What is the amount of effort required to interpret the physical state of the system and determine whether intentions have been met?
3. Can the person determine the functions of the device unaided?
4. Can the person tell what actions are possible unaided?

• Thought that everything was interactive
  • Make the line graph on the calendar bigger
  • Shrink the size of the calendar

1. Can the person determine the mapping between intentions and physical movement unaided?
2. Can the person perform intended actions unaided?
3. Can the person tell if the system is in the desired state unaided?
4. Can the person determine mapping from system state to interpretation unaided?
5. Can the person tell what state the system is in unaided?
6. Can the person determine the relationships between actions and results?
7. Can the person determine the relationship between the controls and their effects?
8. Can the person determine the relationship between the system state and what is visible?
9. Can we adjust the arrangement so as to reduce the number of labels while improving c comprehension (i.e., can we find natural mappings)?
10. Can people muddle through without reading the instructions or help from the experimenter?

Part 3: Goal-directed tests

1. Can you think of something you'd like to learn from the device? If you wanted to learn that, what would you do?

• Reminded him of the PNC calendar, so he could learn about his usage in a similar way he learns about his spending
• Learn from the color coding
• Learn about the billing cycle of each month

1. Can you access the calendar?

• Yes from the bar graph

1. Can you show a bar graph of monthly consumption?

• Yes

1. What is the monthly bill?

• $90~

1. How much electricity is being used right now?

• 43.28 kWh -- wrong answer, but confused this with total monthly cumulative or "Power" on the bottom

1. What is the current price of electricity?

• $.13

1. Can you switch from calendar to bar graphs and back?

• No

1. How many kWh were used yesterday?

• 22.7 kWh

1. How many kWh are projected to be used tomorrow, next week, month...

• $120

1. Can you switch from bar graphs to pictures?
2. Can you switch from pictures to bar graphs?
3. Can you switch from calendar to pictures?
4. Can you switch from pictures to calendar?
5. Can you switch from bar graphs to calendar?
6. Can you switch from calendar to bar graphs

Part 4: Trunk test

Have participant close eyes and take them to randomly selected pages (actually use a random number generator for this). Can they answer the following questions?

1. What is this?
2. What page am I on? (page name)
3. What are the major sections of this site? (sections)
4. What are my options at this level? (local navigation)
5. Where am I in the scheme of things? (“you are here indicator”)
6. How can I get back to pictures and the default page?

Part 5: In the dark test

Turn off the lights. Secretly give them the control upside-down. Can they still use the device?

Part 6: Dual task

Can participants dual task (e.g., n-back) and also use the IHD?

Part 7: Destructive Testing

Can they break it? Not physically.

Part 8: Open-ended suggestion period

• Don't want to see tips every time the device is used
• Don't wand noise or sound reminders
• Cycling through the same tips will be annoying and not be read
  • More likely to read if randomized after going through the first set of tips sequentially
  • More tips are better than less
• Need explanation of high usage days
• Need cumulative kWh
• Would place the device next to his thermometer because he tries to use as little AC as possible, so it can serve as a reminder
• Willing to use, but not WTP upfront or subscription
• Would use if it came from the utility
• Would be less likely to use if it came with advertisements
  • Gave example of his wife's kindle and how it displays an ad every time it turns off
• Would rather pay for renewable energy blocks than pay for a device to reduce consumption

Navigation

• Go back to 2012/07/06
• Notes for Technology Usability Testing (IHD_field_trial)
• Go back to Technology Usability Testing
• Go back to IHD_field_trial