MEMORANDUM

To: Name
From: Kyle Miller
Date: March 10, 2017

Subject: Results of User Testing on How to Make Soap with Coffee

Introduction/Objective:
The goal of user testing the document How to Make Soap with Coffee is to determine the readability of
the document as a whole from the perspective of three readers of varying skill. The testing process
consisted of a location test and a follow-up interview. The tests produced information suggesting that the
layout of the document is acceptable for novice through advanced users, as well as provided several
suggestions for improved wording and necessary graphics.

Method:
This section contains descriptions of the users tested, their significance as a tester, the tests used, and the
relevance of both tests.

Users:
User One:
o Experience Level: Intermediate
o Experience: Enrolled in a university-level chemistry lab, familiar with preparation and
handling of caustic and noxious substances. Also experience in technical writing and peer
review.
o This user was chosen to provide criticism on the structure and clarity of the document.
User Two:
o Experience Level: Intermediate/Advanced
o Experience: Studying for a B.S. in Pharmacy, enrolled in university-level chemistry labs,
extremely familiar with the handling of equipment, comprehension of the saponification
process. Also experience in technical writing.
o This user was chosen because of their advanced background in lab work and chemistry
knowledge. Their ability to provide input on descriptions and visual cues to improve a
novice users experience is essential to a successful revision of my instructions.
User Three:
o Experience Level: Novice
o Experience: No prior soap-making or lab experience
o This user was selected to test the usability of the instructions for a novice. With no prior
experience, the information gathered allows for novice-friendly revision.
2 Name

Method:
The user testing was broken into two sections, a location test and an interview. The location test was
focused on locating key steps, terms, and safety information. The test was introduced with a brief
explanation of the process, including that they would be timed, that they refrain from reading the
document between questions (to prevent excess familiarity with the layout), and that they would be asked
for step numbers and information. The user was placed to the side of the tester, in front of a laptop opened
to the document. No assistance was offered other than repetition of the question. The process is as
follows:
Process:
1. Read the question aloud to the user.
2. Start stopwatch from 00:00.00
3. As an answer was given for each question, mark with a check if correct or X if incorrect.
No chance to answer again if originally incorrect.
4. Mark time upon the user answering each part of the question.
5. Reset stopwatch to 00:00.00
6. Ask following question.
7. Repeat remaining questions.
Upon completion of the location test, the user was given the option to read the document at their own
pace. Following the reading, an interview was completed. The user was informed that the interview was
not timed, that they may refer to the document at any time, and that they may stop testing at any time. The
goal of the interview was to discern user confidence in using the instructions as is as well as to have
them suggest specific areas of improvement in future versions of the document. During the interview, the
user was sat next to the tester in front of a laptop opened to the document. A series of 8 questions were
asked, with discussion of each question if deemed necessary by the tester. Responses were recorded on a
piece of paper, as well as any noteworthy comments made by the user.

Results/Discussion:
While none of the users had experience making soap before, skill level was decided by lab experience and
understanding of the chemical process involved in soap-making. This alone may make a direct conclusion
difficult, as individuals who have no bench skills may be unable to successfully use the document. An
attempt to correct for this lack of experience was made through the careful selection of the novice user, as
they lacked both soap-making and lab experience.
The location test provided information regarding how quickly users could scan an unfamiliar document
and find important steps (Figure 1). While the questions were carefully selected to force the reader to
randomly scan through the document, the test was not perfect as average location time decreased as the
test progressed, showing the users gaining familiarity with the document. Overall, the document is
acceptably readable, as the longest average response was 93 seconds.
Overall, the users were confident in the ability to use the document without graphics to successfully make
soap, as shown in interview questions 1 and 7 (Figures 2 & 3). Their confidence was not without
concern, as several suggestions were made. The most important suggestion was made by users 2 and 3,
asking if the ingredients were measured by weight or volume. Question 4 asked the users to suggest
pictures they would find useful in an updated document. Each user mentioned their interest in seeing
3 Name

pictures of the soap undergoing trace and how to prepare the mold. User 2 also suggested showing what
the soap looked like with coffee in the trace as well as how to liquefy the palm oil.

Conclusion:
In conclusion, the only major changes the document will need to undergo is the inclusion of pictures.
Additionally, it will have to be made clear that the measurements are by weight instead of volume.
4 Name

Appendix:

Figure 1. Chart of response time

Figure 2. Table of responses in interview

Questi User 1 User 2 User 3
on
1 Yes Yes Yes
2 Where to purchase Is dry lye caustic? None
lye? Where to add Spoon type? Add
fragrances? troubleshooting.
3 None Weight or volume? None
Cooling process? What
is freezer paper?
4 Show soap mold and Soap during drying Picture of trace and
trace process process, wrapping, preparing mold
grounds in trace, lye
into oil, burping?, lye
into water,
measurements,
melting palm oil
5 No No No
6 Usable Abbreviations change Switch part 2 and 3
bolding, seems to be
written for those who
have lab skills
7 Yes Yes Yes
8 No Mention not reheating none
5 Name

the lye, materials on

top is a good idea,
add tips

Figure 3. Test and interview questions

Test 1: Location Test
1. List the safety materials required, and the step (part, then number) they are
told to be worn.
a. Time:
b. Correct:
i. Gloves
ii. Goggles
iii. Materials, Part 5 Step 1, Warnings
2. What step (Part, then number), will the mixture begin to trace.
a. Time:
b. Correct:
i. Part 6 Step 5 or Part 6 Step 5
3. What is the ideal temperature range to keep the melted oils at?
a. Time:
b. Correct:
i. 120 to 130 degrees Fahrenheit
4. When mixing the oils with the lye solution, what is the ideal difference in
temperature between the mixtures?
a. Time:
b. Correct:
i. Within 10 degrees Fahrenheit
5. What step (Part, then Number) describes the container used to measure the
lye crystals?
a. Time:
b. Correct:
i. Part 6 Step 5
6. What part outlines the amount of time it takes for the soap to fully cure? How
long does it take the soap to fully cure?
a. Time:
b. Correct:
i. Part 7
ii. 4 to 6 weeks

7. What is the minimum amount of curing time recommended before the soap
can be used?
a. Time:
b. Correct:
i. 3 to 4 days
6 Name

Interview:
1. If provided with the necessary materials, are you confident in your ability to
successfully make soap?

2. After fully reading the instruction set, were there any questions you had
about the process? If so, what were they?

3. Were there any instructions that were unclear or difficult to understand? If so,
what were they?

4. If pictures were included in an updated version of the instruction sheet, what

would you like to see in them?

5. Are there any steps that could be combined or omitted? If so, what are they?

6. What is your opinion on the overall utility of the document? Instructions,

format, legibility, etc?

7. Would you be confident in using these instructions as-is to make soap with
coffee?

8. Are there any comments you feel are important that were not addressed in
the previous questions?

Figure 4. Raw data artifacts

7 Name
8 Name