Take me to findings
Background & Purpose Seeking user feedback on 2 alternative design concepts (Shared Input & 2 Input) for the Payroll Calculator, a product that is intended for individual employees or payroll professionals to enable them to “envision” their paycheck.
In ADP language this is considered a “RESTful API that is used to perform stateless and anonymous paycheck calculations based on user provided data.” The focus of the feedback sought was to address which of the 2 concepts resonate best with users for basics like:
|
Recruiting Users
More than 450 people visited the survey site. 15 users were recruited and ultimately 12 participated in the A/B testing. An incentive of $25 was paid to the 12 users who participated in the test |
Methodology & Technology
The study was conducted using “remote testing” employing a screen sharing technology (GoToMeeting as BlueJeans does not currently support "remote control" of keyboard and mouse by another user). The stimulus was run on the researcher’s machine and the user given control. A "talk out loud" protocol was employed asking the user to provide "play by play" commentary as they engaged with the prototype and undertook a scenario given to them by the researcher.
Here is an outline of the protocol that was used.
Protocol Outline
Scenarios were randomized across the 12 respondents.
Scenario 1
You recently heard on the news that President Obama and Secretary Perez from the Department of Labor announced “The Final Rule” which introduces new regulations regarding overtime pay. These will be effective December 1st, 2016. Your paycheck may change and you want to see how your paycheck might look in the future. Your Salary is $45,000 a year. You qualify for overtime pay for any hours beyond 40. You typically work 60 hours in a week, meaning that you are entitled to overtime pay for 20 hours each week @ $22.50 per hour
Scenario 2
Congratulations, you have landed your dream job! You will be moving to Arizona and be receiving a 50% increase in your salary. You want to see how your change in location and salary will impact your take home pay.
Scenario 3
Your boss has offered you an opportunity to work part time on an hourly basis in addition to your current salaried position. You will now be receiving 2 pay in 2 different ways and want to see what that will look like.
Here is an outline of the protocol that was used.
Protocol Outline
- Intro (2 Mins)
- Tell me about yourself and your work (user background) (3 Mins)
- Scenario
- Present Concept 1
- Gather ratings & Explanations
- Same Scenario
- Present Concept 2
- Gather ratings & Explanations
- Wrap Up
Scenarios were randomized across the 12 respondents.
Scenario 1
You recently heard on the news that President Obama and Secretary Perez from the Department of Labor announced “The Final Rule” which introduces new regulations regarding overtime pay. These will be effective December 1st, 2016. Your paycheck may change and you want to see how your paycheck might look in the future. Your Salary is $45,000 a year. You qualify for overtime pay for any hours beyond 40. You typically work 60 hours in a week, meaning that you are entitled to overtime pay for 20 hours each week @ $22.50 per hour
Scenario 2
Congratulations, you have landed your dream job! You will be moving to Arizona and be receiving a 50% increase in your salary. You want to see how your change in location and salary will impact your take home pay.
Scenario 3
Your boss has offered you an opportunity to work part time on an hourly basis in addition to your current salaried position. You will now be receiving 2 pay in 2 different ways and want to see what that will look like.
You can download the discussion guide via link below:
discussion_guide_ab_pr_calculator.pdf | |
File Size: | 1073 kb |
File Type: |