eServices for Citizens is the name of the government unit that is responsible for public-facing digital services. Part of the work we do is to conduct assessments throughout the service delivery process. We do this to make sure the work teams do aligns with the government's digital service standards. This means they have been designed to create a consistent, positive user experience for citizens.
If, through this process a piece of work is not aligned, we work with project teams to create that alignment.
The current process
Once eServices meets with a project team in Pre-discovery, we determine which assessments the team will need to complete and pass before they can launch the service to the public. Factors that influence this decision are the size and complexity of the project, the timeline and sometimes the budget. Assessments are timed after Discovery, Alpha and mid-Beta.
These assessments are currently called Service Maturity Assessments.
How we conduct Service Maturity Assessments
The User Experience Manager leads service maturity assessments. They meet with the government project team and sometimes external contractors that have a role. The idea is to get people who fill the project team roles and represent different areas of expertise in one room or on one call.
Together we work through a series of questions on the following topics.
- User needs and user experience (UX)
- Service quality and compliance
- Agile methodologies
- Project ownership and accountability
- Service adoption planning
- Launch planning
- Transition to solid state
Once the assessment is complete, the User Experience Manager completes a report and shares it with the team. The report identifies potential risks to launch and provides the team with requirements to meet and steps to meet them.
The final step in this process is for the Service Delivery Manager to run a Pre-launch Checklist on the service as the final compliance check.
Levelling up the assessment process
We are constantly evolving our processes to stay up to date with industry and other jurisdiction best practices for organizing and executing this work, spread the workload across the team and make it as easy as possible for departments to understand the reports and meet compliance.
Over the winter we looked at all of these factors and began executing our plan to improve the assessment process.
Staying current with industry and other jurisdiction best practices
The UX manager met with colleagues from across Canada to learn more about their processes. One thing we have heard from a few teams we've partnered with is that there are too many assessments and they feel repetitive. We've updated the current version several times and the assessment sessions went from 1.5 to 1 hour in duration. We also combined Discovery and Alpha assessment as an option for teams that were somewhere in the middle of the phases when they met with us. Rather than make them go back and redo work, we keep them moving forward.
When we met with other jurisdictions we learned that many do 2-3 more assessments then we currently do. They have a more formal Pre-discovery assessment, 2 assessments in Beta (mid and end of beta) and after the digital service goes live. This process works for them because they have a team of people to conduct the assessments, review and create reports and they can send teams to get support to meet the requirements they've fallen short of meeting.
After these meetings I was confident we should not reduce the number of assessments we do. As it is, we have a streamlined process and we have flexibility to reduce the number of assessments a project goes through. For example, if we work with a project manager who knows the service delivery process in and out and they consistently meet the digital service standards - they would not be subject to as much rigor. Or they might take the first crack at filling in an assessment so the meeting takes less time.
It was incredibly invaluable meeting with these teams across Canada and having an opportunity to share what we do and learn from them.
Spreading the workload across the eServices team
eServices is a small team. Every staff member has a specific role, but we also have a lot of expertise and it's not strange to see us working outside out job descriptions. It's one of the things we like about our jobs and it keeps things interesting.
At the moment the UX Manager leads these assessments, writes the reports and with the Service Delivery Manager works with the team to tackle compliance issues. This is for all public-facing digital service projects. It means we can't do more than 2 assessments a week. We do everything we can to avoid it, but there is a potential we create a bottleneck and add to a project timeline if we can't complete this work efficiently.
The question is, how do we increase our capacity to get projects through the assessment process as efficiently as possible? To do this we ultimately landed on pulling out the user experience pieces into a separate assessment focused on pieces related to the user experience. These assessments are conducted by the UX Manager and they're called:
- After Discovery UX Assessment - scheduled after the team complete Discovery and before they move on to Alpha.
- After Alpha UX Assessment - scheduled after the team complete Alpha and before they move on to Beta.
- After Beta Assessment - scheduled after the team complete Beta and before they launch the digital service to the public.
The remaining pieces formed what we're calling the Service Compliance Check. This is a checklist the Service Delivery Manager completes throughout the service delivery process. Service Delivery Managers conduct these checks with project teams and keep track of their progress. These checks focus on things like security, privacy and change management.
Once we had our two lists, we went to work to improve the project team user experience.
Making it easier to interpret assessment reports
The current format for service maturity assessments is a Word document. By the time we include screenshots and requirements, it was be a very long document. And after several iterations, it wasn't clear to some clients that they had to meet the requirements to pass the assessment.
Scoring system and explaining how we assign scores
For the UX assessments, the first thing we did was create a scoring system. I'm very thankful to the Government of Ontario team who shared their expertise and documentation with me because it really guided my work. There were two pieces that really helped. The first was a scoring rubric they share with project teams and the second was how they assigned scores for compliance.
Inspired by their scoring rubric documentation, I set out to create something similar for the Government of Yukon. I think project managers will appreciate the preview so they can see where a project has to be to pass an assessment.
Next I started defining how I score the requirements. Some of the wording may change before we fully launch the new assessments but as of today we have:
- Approved - this means the team met the requirement.
- Course-correct - this means the team hasn't quite met a requirement and they have some work to do before the next assessment.
- Stop - this means a team missed a critical piece of work and we need to stop for a moment to come up with a plan together so we can tick that box.
Evolving our report format
Next, we updated the report format. Rather than a lengthy Word document we have a spreadsheet that we use to convey the status of each line item. We're in the process of testing this out with clients and I imagine there will be a few updates before we roll them out to more teams.
When we'll launch the new assessments and checklists
All of this work has culminated in testing out the new format with select clients and learning how they work for them. Over the summer we'll refine the designs and we're aiming to roll them out this fall. You can expect another blog post about this and a series of updated pages in the Digital Service Delivery Guide as well.