Objective:
The Public Service Commission (PSC) requires the ability to target key user segments using demographics, and to conduct user testing to baseline performance and validate design decisions of improvements under consideration. In addition, the PSC requires the capability to conduct relevant online user experience testing on reliable sample sizes of key user segments within short timeframes.
Acquiring this capability reduces the cost and time per test and provides the PSC with a scalable capability that can be used for all user validation of online services.
The objective of this contract is to acquire an online User Testing Services that will provide these capabilities to the PSC.
Background:
With millions of visits a year, the jobs.gc.ca web page is among the busiest in Canada. As a result, the PSC responds to over 60,000 annual inquiries related to those visits. Continuous improvement of the jobs website depends, in part, on user analytics and the need to validate improvements under consideration with segments that represent the PSC’s existing user base to avoid costly design and maintenance defects.
Services and innovation Branch (SIB) is accountable for the adoption of innovative solutions and practices for the continuous improvement of user performance. It is also responsible for implementation and ongoing improvement of a User Experience (UX) Center of Expertise and Tiered User Support Model. As part of these deliverables, continuous improvement depends on both top task reporting on user support and web usage depend on timely, accurate and reliable analytics as well as the ability to quickly validate design decisions with users who represent the PSC’s user segments.
In order to implement user improvements in a manner that is relevant to user needs, efficient in terms of response times, and reliable in terms of validity of testing results, the PSC requires access to a service offering that combines these three capabilities into a single simple accessible service.
Scope / Tasks:
The PSC requires the services of a contractor to provide the capability to conduct relevant user experience testing on reliable sample sizes of key user segments within short timeframes, which can be used for all user validation of online services.
The work will involve a subscription to Online Testing Services, providing the PSC with the ability to:
select from a group of experienced user experience testers using demographics that represent PSC user segments;
quickly perform user experience testing and obtain baseline testing results on the Public Service Resourcing System (PSRS) live system to focus on causal analysis behind poor performance or to validate any design improvements under consideration; and
provide guidance on the improvement of user experience testing service delivery, the development of user experience tests, and the application of user experience testing results in the continuous improvement of PSRS service delivery.
To provide the above, the contractor will, as part of the Online Testing Service:
provide PSC with access to its own large custom panel of users that accurately reflects the user population of the jobs.gc.ca website;
include English Canadian and French Canadian users;
test with its own users (beyond those in the panel), based on a list of users to be provided by the PSC;
perform moderated testing, whereby either the service provider or the PSC can interact with users during testing sessions. The PSC requires support from the service provider to facilitate usability test sessions within a very limited time period (ex. less than one day);
access and manage UX tools, and to run additional tests, above and beyond remote usability testing sessions (for example, navigation testing or one-click testing tools);
provide access to additional analytics or data (above and beyond the PSC’s own analytics tools) gathered through the testing sessions;
provide support to analyze usability test data within a very limited time period (ex. less than one day);
provide analytics related to tests run on the services;
provide and combine testing types to send to users at the same time as part of one test initiative (ex. combining a remote unmoderated test of a live site with a one-click test of prototype);
perform testing with users that have access to the Internet via desktop or mobile phone.
