Webinar attendees: If you didn’t get your questions answered during the live webinar, scroll down to the bottom of this post for a full Q&A session with Carol Dibert!
In our recent webinar, Skilljar’s VP of Marketing, Michael Freeman, hosted a discussion with Carol Dibert, Manager, Certification at Zendesk. Carol shared the secrets of launching and running a successful certification program based on her years of experience with SaaS companies such as Autodesk, Salesforce, and now Zendesk. From determining if your company is ready for a certification program, to deciding when and how to launch it, to recommendations for getting the most out of your investment in certification – Carol is a true expert willing to share her wisdom with learning professionals.
View the on-demand webinar or read the recap below.
Getting Started With Certification
How can you know if the time is right for your company to offer a certification program?
Carol suggests you examine your reasons for wanting to offer certification. Is it to:
- Offer a program for individuals to be able to differentiate themselves in the job market, based on your particular area of expertise? For the individual, this helps other companies understand that their product skills have been validated by your company.
- Create a staff well trained in meeting the needs of customers to provide a better customer experience? If your customers have a team certified in your product, they can tend to the needs of their own customers, creating less support tickets for them, and ultimately for your product as well. Ensuring you have professionals in the field configuring and deploying your solutions correctly, and with expertise, ensures the whole ecosystem is focused on customer success.
- Demonstrate your maturity as a company, one that is proliferating solutions in the market? When you’re at a point where you can offer a certification program, you’re really telling the market and your shareholders that you’re ready to unleash a force of professionals implementing your products and solutions, leading to greater awareness and proliferation of your offerings in the market.
All of these reasons are valid, and all are reasons why Zendesk devoted resources toward establishing a certification program. Another reason you may want to develop a certification program is to promote awareness for your company as a leader in its space. This may be a useful tactic if your competitors are not yet offering certifications — you can establish yourself as the leader and attract more attention for your brand.
Training, Certification, and Badging
Sometimes, these words are used interchangeably but they are really three different things.
As Carol explained:
Training refers to ungated content the user consumes on their own time and at their own pace to learn something and pick up skills, feature by feature, with no real commitment (unless it’s required by an employer). Training is about learning for the individual, and teaching for the company.
Certification is a measurement of skill, how well an individual can apply their learning to a total solution. Certification isn’t teaching and it’s not feature-specific, but validates the knowledge of a set of features that can enable the user to implement a solution, such as becoming a Zendesk-certified admin. Certification requires a commitment; there are consequences for not showing up.
Badging is about validating what a user has learned along the way to a greater track of training courses. At the end of the course, a user takes an assessment to make sure they retain the information from the course. If they pass that assessment, they get a badge. Badging serves as a reward for the user accomplishing something and encourages them to continue their learning toward full certification.
Related: Moving to a Hybrid Workforce? Why You Need to Offer Online Certification
What Do You Need Before You Start a Certification Program?
Ready to offer a certification program for your product or solution? Not so fast. Carol suggests companies explore these three considerations before deciding to move forward with a certification program.
- Corporate Buy-in Do you have the support of the C-Suite and other internal teams that you will need to get your program off the ground – such as Product, Customer Success, and even Marketing? It’s important to make a case to management that certification can move the needle on the reasons for offering certification identified above.
- Budget Corporate buy-in leads to…you guessed it…money to build exams and invest in technology to provide a great test-taking experience. Having a learning management system (LMS) from the get-go will make your life so much easier. Hopefully, when you’re starting up your certification program, you’ve got a training team in place that is already leveraging or planning to leverage an LMS.
- Training Content So you’ve got corporate buy-in and budget – congratulations! But you still can’t launch your program without training content. Carol advises working with an Instructional Designer or some other form of training support — whether they’re on your team or an adjacent one — because training and certification go hand in hand. This partnership will be invaluable as you map out your blueprint for certification.
For Zendesk, moving certification into Skilljar as our LMS was the best way to bring a good customer experience to our certification program. — Carol Dibert
Related: Building a Business Case for Customer Training
Questions To Ask Before You Start a Certification Program
So how do you know if you’re ready to start a certification program? Carol proposes asking the following questions:
- What is the purpose of your training? See first section above.
- Is your user base ready for it? If you build it, will they come?
- How many people can you expect to reach? You need this information to justify getting corporate budget and buy-in.
- How long have they been using your product? What does the training data tell you about usage patterns?
- How long have your products been out there? You wouldn’t want to create a certification for a new product because no one has learned how to use it yet.
- Is your product mature enough to have a depth of content to write questions against? Keep in mind you might need to create twice as many questions that ultimately make it into your exam after they are vetted by your training support team.
- How volatile is your product? What is the likelihood of features changing quickly? This could lead to complications down the road in having to frequently update trainings in your certification program.
In short, you want to make sure that it’s the right time in your company’s maturity for reaching the right amount of people to offset the costs and time required to implement a certification program.
The Right Integrations Make Certification Easier
We meet up with companies like Slack, Smartsheet, and DataRobot (all Skilljar customers) to help each other plan and manage our certification programs. Everyone might be at a different stage, but this is part of the revolution that’s coming with certification, especially for SaaS companies. We move so fast, the traditional ways of working just don’t apply anymore. — Carol Dibert
Zendesk uses a number of tools that all work together — all of which integrate with Skilljar — to get their certifications up and running and ensure the best customer experience.
- Examity is an online proctoring program that ensures the user is actually taking the test in a virtual environment within the allotted amount of time, without consulting external resources, i.e. Google.
- Caveon ensures exam security so that tests are fair, reliable, valid, accessible, and accurate.
- Credly allows you to create, issue, and manage digital badges.
And of course, all of these are compatible with Salesforce (also integrated with Skilljar), which ensures Zendesk can easily tap into an existing pipeline of information on their customers.
We just expanded on the integration that already existed with Salesforce so all the information we needed from our tool stack could be exported from one foundational location. — Carol Dibert
Expert Tips from Carol Dibert:
- Don’t skimp on online proctoring for certifications in the age of digital learning. As a company, it’s going to do damage to your reputation later on, if you grant certifications to people who have not legitimately taken and passed your training.
- Be sure you work with an Instructional Designer (ID) to vet your training blueprint. IDs do audits of our trainings to see if there are any nuggets in the training blueprint that are missing, and then plot out how they can fill those gaps. This ensures all our bases are covered.
- Cultivate relationships with your Product Teams. Get their input on what product might be first up for certification. Use them as your stakeholder and have them help you identify who might be subject matter experts for you as you develop your training content.
While this may sound like a lot of material to consider, it’s only the tip of the iceberg for what Carol covered with us. View our on-demand webinar to learn more. But most importantly, heed this advice from Carol once you’ve decided to offer certifications for your product:
Just get started. It’s not going to be perfect and you’re going to iterate over time, but you have to just do it.
View the on-demand webinar here!
Post-event Q&A session with Carol Dibert
The response to our webinar with Zendesk’s Carol Dibert was quite impressive – many of you are clearly interested in the topic of certifications! Since we weren’t able to answer all of the questions you asked during the live event, we’ve captured your questions here for Carol to address:
Q: How do you proctor certifications to scale?
Q: When you talk about the test/exam, do you typically just use a multiple-choice format, or do you leverage a practical exam where users are actually in the software?
Q: Should certification exams be multiple-choice (e.g. 60 questions) but ALSO proctored from a knowledge demonstration perspective? Like a written test and a road test? That is our plan, where we won’t proctor the written test but WILL proctor the road test, somehow?
Q: How can we check who has posted their certification on LinkedIn?
Q: What are the high-level steps to update an existing certification?
Q: Can Skilljar handle the background of showing someone who clicks on a badge to know what was required to earn the certification and the person’s standing related to that certification (e.g., expired, etc.)?
Q: What are some of your favorite features of Skilljar that you think companies should leverage?
Q: Do you have SSO from your actual Zendesk product and Zendesk Training (Skilljar)?
Q: Who does the proctoring? The education team?
Q: Thoughts on requiring customers/partners to pay for SaaS certification? If so how much?
Q: Does Examity handle proctoring details such as (a) 40-min time limits for multi-choice questions, (b) showing answers AFTER the exam is submitted, (c) demonstration of knowledge proctoring, etc.
Q: Are all of Zendesk’s certifications live? Do you have an on-demand preparation option?
Q: Can there be a combined training & certification program (learning and then measuring)?
Q: How do you proctor remotely?
Q: Can you address the difference between certifying product use vs industry skill, for example “how to use Zendesk” vs “how to effectively support customers”?
Q: What are your thoughts on AI proctoring?
Q: What is the value of American National Standards Institute (ANSI) recognition for a certification? Is it necessary or “just nice”?
Q: Is there any way to reconcile certification with a product that is incorporating new features at a high pace?
Q: Is there a calculation you use when determining when it’s time to update the certification or require new certification?
Q: As a SaaS company that issues new features throughout the year, how does that impact your recertification?
Q: We only have bandwidth currently to offer low-stakes certification. Is there a standard for how many questions one should include?
Q: Is that meetup you have with Slack, Smarsheet, etc. open to any companies to join?
Q: How do you keep up with the updates in the SaaS world and the Certifications?
Q: Two questions if we are selling certifications to external customers a) How to show value of certification program to external customers b) How to price it?
Q: In an effort to just get started and not having any budget – would you recommend starting without a proctoring option and working our way into that?
Q: When you talk about iterating – how do you communicate big changes in a certification? ie. Our current certification is not proctored and we want it to be – should the old one become invalid?
Q: Could the list of tools being used be noted somewhere?
Q: Do you have a recommended template that you have Subject Matter Experts complete when they are helping the Instructional Design team complete a Storyboard on a particular topic/lesson/certification?
Q: What’s the benefit to having a Credly vs. using Skilljar’s native badging?
Q: How do you proctor certifications to scale?
A: When using an online proctoring vendor like Examity, you pay a fee for each exam proctored. Fees are typically assessed based on how long your exam is and how many testers you anticipate will sit for your exam annually. The vendor carries the burden of supplying the proctors and support for your testers during the testing process.
Q: When you talk about the test/exam, do you typically just use a multiple-choice format, or do you leverage a practical exam where users are actually in the software?
A: Exam formats and exam question formats differ based on your testing needs. Multiple choice questions are pretty common because they are versatile. You can ask a straightforward type of recall question, you can set-up a scenario-based question that requires critical thinking or you can offer a performance-based question where they demonstrate skill in the software. But there are lots of other question types to choose from too such as matching, discrete option multiple choice, true/false, smart item, etc.
Q: Should certification exams be multiple-choice (e.g. 60 questions) but ALSO proctored from a knowledge demonstration perspective? Like a written test and a road test? That is our plan, where we won’t proctor the written test but WILL proctor the road test, somehow?
A: It sounds like you have a multi-level certification opportunity here! Perhaps you could offer a proctored 60-question multiple choice exam as a level I exam and as a required prerequisite to a level II exam. The level II exam would be what’s called a performance exam where a scenario is given for the candidate to prove/defend/demonstrate to a panel of judges based on a rubric. Be careful with performance exams. They are very staff intensive. Here’s an example of one Salesforce offers to give you an idea of what’s involved.
Q: How can we check who has posted their certification on LinkedIn?
A: There is a Licenses & Certifications section on individual LinkedIn profiles that you can scroll down to see but you’d have to look people up one by one. Alternatively, if you want to see who is Zendesk certified, you can use our talent directory and search by name, skill, or product!
Q: What are the high-level steps to update an existing certification?
A: Working with SMEs to review your blueprint looking for any out-of-date or missing objectives. From there, reviewing the stats on your entire item bank to identify any poor performing items and then going through all of the items with your SMEs to find and out-of-date content. Once you understand how much you need to update you can determine what approach is best for your audience. That could mean introducing some unscored items to your test form until you collect enough stats to replace bad items permanently or it could mean creating a delta exam. For some smaller shops, they may choose to fix items on the fly and monitor stats as they come in and make adjustments as needed.
Q: Can Skilljar handle the background of showing someone who clicks on a badge to know what was required to earn the certification and the person’s standing related to that certification (e.g., expired, etc.)?
A: The badge metadata is actually managed on the badging vendor’s side which in our case is Credly, but since Credly integrates with Skilljar, our learners are able to see their badges in their Skilljar learner profiles too.
Q: What are some of your favorite features of Skilljar that you think companies should leverage?
A: My favorite feature is the ability to have learner certification achievements housed in the learner profile with all of the other learning achievements like course completions, quizzes, learning paths. It let’s the learner track their entire learning journey from start to finish in the same place! I’m quite a fan of “pathing”. I like the idea of recommending a track of courses. I also use Skilljar’s reporting quite a bit and find it super easy to work with.
Q: Do you have SSO from your actual Zendesk product and Zendesk Training (Skilljar)?
A: This one is a little out of my wheelhouse. I leverage Skilljar for certification but I am not the Skilljar Product Manager or Administrator. I can tell you that we employ SSO between Skilljar and Examity so that when a learner registers for an exam on Skilljar they are taken seamlessly over to Examity to get it scheduled. If you need an answer specifically about SSO from Zendesk to Skilljar you can message training@zendesk.com and someone on the team will get back to you with a proper answer.
Q: Who does the proctoring? The education team?
A: We use Examity to remote proctor our exams. They have many proctoring options from live humans to full AI and anything in between. We are using live humans at a 1:1 ratio (1 proctor to 1 student). On the occasion we run a live event, we use our own team to proctor our testers in person (which we haven’t done in quite some time due to COVID and friends).
Q: Thoughts on requiring customers/partners to pay for SaaS certification? If so how much?
A: To me, all SaaS certifications should be offered at a price. For one, they cost money to create whether you use internal staff or not, there is a cost. They also require ongoing maintenance so you’re looking at recurring costs (resourcing and license fees). What will you be measured on? Revenue? Engagement? Both? There is also a psychological component to charging – the perception is that if it’s free it isn’t any good. The free promo codes we have given for certification have been used the least and have the highest fail rates.
Q: Does Examity handle proctoring details such as (a) 40-min time limits for multi-choice questions, (b) showing answers AFTER the exam is submitted, (c) demonstration of knowledge proctoring, etc.
A: Examity does collect the proctoring details from you for your exams such as duration, rules, and so on. We do not show answers after the exam because then your exam bank is exposed and you are at risk of a breach but I suppose if you wanted to you could make those arrangements with them. If by demonstration of knowledge proctoring means do they proctor or judge performance exams that use a rubric, I’m not sure. We do not use them in that capacity so you’d have to inquire with them about that.
Q: Are all of Zendesk’s certifications live? Do you have an on-demand preparation option?
A: All six of the Zendesk exams are online proctored and can be scheduled 24/7. We have learning paths and certification prep paths for each and for the Zendesk Support Administrator Expert and Zendesk Explore CX Analyst Expert exams specifically we have virtual live prep & practice workshops.
Q: Can there be a combined training & certification program (learning and then measuring)?
A: I might be missing the intent of the question here but that is exactly what we hope we are offering. Our learning paths finish up with a non-proctored knowledge check to make sure you are retaining what you need to proceed. We also have non-proctored “mock” exams so you get some practice in without any of the pressure. When you feel ready and eventually sit for the certification exam, upon completion you will receive a Pass/Fail result along with a topic-level score report that will show you how you performed in each section of the exam. This way you can see how well you know or don’t know each area of the exam.
Q: How do you proctor remotely?
A: We rely on our proctoring vendor, Examity, to proctor our exams for us. They have proctors available globally that allow our testers to schedule an exam 24/7. Using Zoom or GoToMeeting they monitor the test taker as they go through the test to ensure the tester doesn’t run into any testing issues and to also make sure our exam data stays secure.
Q: Can you address the difference between certifying product use vs industry skill, for example “how to use Zendesk” vs “how to effectively support customers”?
A: Certifying a product or solution use is somewhat finite because those skills are specific to a job where a task is generally done a certain way or can at least be identified from subpar ways. Certifying soft skills is less defined and not specific to a job because soft skills are universally used which adds a layer of complexity.
Q: What are your thoughts on AI proctoring?
A: AI proctoring provides advantages with regards to lower pricing, and supporting global scheduling but depending on how advanced it is, it might not be able to interrupt a testing session if cheating is detected. This means your whole test form could be exposed before you even find out someone cheated whereas a human proctor can disconnect the tester from the exam as soon as cheating is detected.
Q: What is the value of American National Standards Institute (ANSI) recognition for a certification? Is it necessary or “just nice”?
A: This isn’t something I’ve seen come up in my experience or with the companies I’ve worked with. I can see maybe a company like Amazon being into obtaining an ANSI certification just due to the nature of their business but not sure it’s necessary for a typical SaaS company offering product certifications.
Q: Is there any way to reconcile certification with a product that is incorporating new features at a high pace?
A: Well, it’s difficult (and costly) to maintain a certification against a constantly changing product. Perhaps you can work with your stakeholders to come to an agreement on a timeframe that you are all comfortable with for having people out there certified but a little out of sync. Is 6 months tolerable? When you come up with an acceptable timeframe it will help you decide the vehicle you use to get testers the new information and how often you charge for it (if at all).
Q: Is there a calculation you use when determining when it’s time to update the certification or require new certification?
A: I hate to sound vague here but it really depends on how much change you are dealing with. If there is a UI change for instance and you have screenshots in 75% of your questions, then you’re looking at replacing a lot of screenshots but if the features haven’t changed, that’s not what I would call requiring a new certification. If an SME reviews your blueprint and says 12 out of 18 of your objectives have had changes and there are a handful missing, then you are looking at redoing the whole thing.
Q: As a SaaS company that issues new features throughout the year, how does that impact your recertification?
A: We are constantly evaluating the impact of feature update impact against our certification content. This is just one of the reasons our relationship with our instruction design team is so critical. Although there is a regular cadence to our exam maintenance schedule where we update content with SMEs, we also monitor item stats more often internally to make sure we aren’t missing anything important in the interim.
Q: We only have bandwidth currently to offer low-stakes certification. Is there a standard for how many questions one should include?
A: If there is a standard, I am not aware of it. For us, we determined 30-ish questions felt right to us for our lower-stakes exams. We arrived at that number by taking a look at how much content was available to write against and also how in-depth we felt we needed to go for a low-stakes exam before it felt like too much. We also tempered the number of questions against what we were offering for our high-stakes exams, which is anywhere from 60-80 questions.
Q: Is that meetup you have with Slack, Smartsheet, etc. open to any companies to join?
A: The meet-up happened organically because we are all participants on a Slack org called Customer Education Org at customered.slack.com. There are channels there specific to all arms of customer education including one specific to Certification. I’d urge you to join up with that Slack channel to make connections and establish a peer group to bounce ideas off of and learn from. It’s been so helpful to me in my career.
Q: How do you keep up with the updates in the SaaS world and the certifications?
A: I network with peers, attend conferences, look for opportunities to collaborate (like this), spend time on LinkedIn, read Certification Magazine and spend time looking at other company’s certification offerings to benchmark what we have and see what other companies are doing in the space.
Q: Two questions if we are selling certifications to external customers a) How to show value of certification program to external customers b) How to price it?
A: The main reasons people get certified is to validate their skills and to gain a competitive advantage in their current job (think bigger, better projects) and/or in the job market. So if you can obtain some customer quotes pertaining either of those things after they take your exam that could be a compelling way to show value.
For pricing I’d spend some time looking at your peer companies and seeing what they are offering and at what price. Start gathering some comparison data and also make sure you consider your own pricing (for training courses or training events) so you don’t cannibalize any of your other revenue streams.
Q: When you talk about iterating – how do you communicate big changes in a certification? ie. Our current certification is not proctored and we want it to be – should the old one become invalid?
A: In the example you give, yes, I would create a new exam to be your proctored exam since the non-proctored exam has been out there ungated. That being said, it doesn’t mean you have to start from absolute scratch. You can clone some existing items or rework them but depending on how long it’s been, it might be time for you to take a fresh look at your blueprint anyway.
Q: In an effort to just get started and not having any budget – would you recommend starting without a proctoring option and working our way into that?
A: Even though it’s a tough sell, I recommend starting with a proctored exam. It takes a lot to create an exam from scratch and if you offer it non-proctored you will just have to spend the time and money revamping your whole exam anyway. It will get breached and put on study sites quicker than you can enjoy your first glass of celebratory champagne. In my opinion, non-proctored exams are just really long knowledge checks. They offer no guarantee that you are backing someone whose skill you’ve actually measured and I don’t think testers value them as much. Again, just my opinion.
Q: Could the list of tools being used be noted somewhere?
A: Learning Management System (LMS) – Skilljar
Test Development – Scorpion by Caveon
Exam Proctoring – Examity
Digital Badging – Credly
Q: Do you have a recommended template that you have Subject Matter Experts complete when they are helping the Instructional Design team complete a Storyboard on a particular topic/lesson/certification?
A: Instructional Design templates are not in my purview as a Certification person. This question is better suited for someone in Instructional Design. Sorry.
Q: What’s the benefit to having a Credly vs. using Skilljar’s native badging?
A: Skilljar is getting there but has a way to go yet for what is needed. Credly is a brand that is widely recognizable, can offer deep metadata options, reporting and consumption analytics, but most of all a talent directory feature. There are other elements related to expirations and recertification that were decision factors for us too.
