Congratulations! You’ve set out to build a new application that will change the world. You spend months cajoling leadership for the funds to build your vision, lead several UX workshops, gather competitive research, and start writing user stories for the developers to pick up. The first few sprints go very well and the team is making progress toward its goal.
During a refinement call, your BA raises an interesting question. Since your product is launching to millions of users, your company has a rule that a piece of code can’t roll out to more than 1m users at once. To get around this, the enterprise architecture team designs the rollout to be completed in a series of waves, which is a common deployment tactic. Somewhat more complicated, though, is the fact that some users on your platform migrated from the old legacy system, while others did not. You have an existing commitment to 300 clients to provide a custom website design to these users which shows information about their migrated plans. In a refinement call, a question surfaces about whether there are any other special aspects of the user experience we have to migrate to the new software to satisfy our client commitments. “Can we ask the person who built the old software?” you naïvely ask.
There’s a deafening silence. A company lifer named Donna, who you’ve never met (yet is somehow attending your team calls), breaks her silence. “I think Milton built that,” she says. “Great,” you reply. “I’ll track him down and invite him to the next meeting.” The following week turns into a game of Where’s Waldo as you pilfer organizational charts and raid old presentation files in pursuit of your White Whale. You finally get in touch with one of Larry’s old teammates, who breaks the unfortunate news: Milton took a golden parachute package last year. He and his red stapler are sitting on a beach in Thailand, happily retired and gone forever.
The company laid off the last remaining person who knew anything about your native system. You’re going to have to figure it out anyway.
What's a Subject Matter Expert?
An SME is someone who possesses advanced knowledge in a particular area, process, practice, methodology, or system. They usually have many years of experience in a specific area of the business which is the source of their authority. Often technical experts, they are comfortable speaking at length about the details and nuances of their area of expertise. SMEs can include employees of a company, contract workers, and consultants. They are commonly articulate, detail-oriented individuals who have spent their careers immersed in a single domain. SMEs often provide documentation on critical issues, break down technical concepts into easily-understood language, improve business processes, make recommendations for future strategy, and answer tons of questions from people all over the organization. They are indispensable members of the team who are often underappreciated and underpaid.
For our purposes, we’ll consider an SME to be someone who has highly specialized, domain-specific knowledge that would be extremely difficult or impossible for the company to outsource. In our case, Larry is the quintessential SME.
Put Leadership on Notice
Communication skills are a hallmark of an effective product manager. Not only does this include the traditional skills of hosting effective presentations, being able to get to the point in meetings, and write effective correspondence, but it also includes knowing when to communicate. For example, not having a dedicated SME on your team is a major obstacle that has the possibility to torpedo your project. If you don’t call out challenges to senior management as they present themselves, then you are by default saying everything is okay. That doesn’t mean you get to run around making excuses about why your project is now facing a delay, but instead you must directly and actively take the initiative to explain that there’s a risk.
As soon as you learn of the risk, make sure management, team members, and stakeholders become aware of it. If you raise the issue high enough, there’s a possibility that other resources inside the company could temporarily be pulled over to your initiative to fill the gap. I’ve seen multiple times where management made a good faith effort to replace the SME on a team by trying to procure free internal resources. It usually doesn’t work, but it’s worth a try. The conversation with management will go like this:
- Explain that there's a major knowledge gap on the team
- Ask whether additional resources can be allocated
- If they cannot, write a mitigation plan
Executives are big-picture people. They aren’t SMEs because they are more comfortable dealing with the forest than the trees. As a PM, you need to be comfortable speaking their language. This may come easily to you if you’re a big-picture thinker already, which many PMs are. In no uncertain terms, you need to explain to leadership that your team is missing a critical SME which will have severe downstream impacts on your delivery timeline if left unchecked.
Pro tip: Break down your argument numerically. If BAs have to make up the slack for a missing SME, this will cost you developer hours. Try to estimate the time overrun and add it to your budget.
Deliver Iteratively
Iterative delivery is a core component of agile for many reasons. It breaks up the work into manageable pieces, keeps the team focused on value delivery, and aligns your work into a predictable cadence. Iterative delivery can also mitigate some of the risk associated with lacking an SME because it lets you iteratively launch individual components of the application, test them, evaluate whether you’ve broken anything.
Here’s an example. During the pandemic, I worked with the American Red Cross as a volunteer product consultant focused on improving an Alexa Skill that they’d built. The Skill was designed to deliver first-aid assistance, quizzes, and facts using conversational AI. My role was to identify areas in the user experience where people were getting stuck or dropping out of the application. In this case, I was taking up the role of SME, in replacing someone who had recently left the organization.
To begin my project, I assessed the analytics. Users were successful in getting to the topics they needed; “Hey Alexa, tell me about bee stings.” But once there, users struggled; “Hey Alexa, show me quizzes.” Accessing individual topics was easy, but navigating between them was hard. I hypothesized this was a navigational issue and presented my findings to the team.
One of the junior product managers suggested he go into the application and fix all of the dialogs that very day. “This will take a few hours,” he said. I cautioned against that approach, warning that without more information, we may be creating new problems. “If we want to solve the problem,” I said, “we should take an iterative approach of A/B testing one chat dialog at a time, reviewing the analytics, and then rolling the change out to users.” My approach was approved and we began to update the dialogs. We changed the first dialog, ran the A/B test, and reviewed the findings. After clarifying the navigational language, our success rate actually decreased. How?
After digging into the subject further, we uncovered that much of the Skill’s user base was international. Users who struggled with English had the hardest time navigating the Skill. When this became apparent, we updated dialogs to be more reflective of users’ native language expectations, and our success rate increased dramatically. I’m glad we weren’t so heavy-handed in our solution. Had we rolled out the change to all users at once, things could have been much worse. Iterative delivery saved the day.
Iterative delivery can be a defensive strategy in situations where you don’t have the necessary experts on your team. Because you don’t know what you don’t know, sticking to small, bite-sized pieces of customer value at a time (in this case, a chat dialog), not only do you have the benefit of mitigating risk, but you also can review analytics in real time. If you release in chunks, you can’t break the whole thing at once.
Reverse Engineer & Wing It
If you’re in a situation where you can’t get additional resources to cover Milton’s departure, and for whatever reason can’t deliver your product iteratively, you may have to wing it. “Did he just say that?” Yes, I did just say that.
"Winging it" is a theatrical phrase referring to impromptu performances given by actors who learned their lines while "waiting in the wings."
Before you launch your product on a hope and a prayer that you aren’t breaking any multi-million dollar systems or leaving millions of users with bricked devices, you will have to make a good faith effort to reverse engineer your existing systems. In our case, reverse engineering is when an organization’s level of subject matter expertise, competence, and morale has dipped so low that it’s forced to hire developers to open the existing codebase and figure out what the heck is in there. Believe it or not, this actually happens.
“Hey, that homemade pizza was delicious.” Do you have the recipe? “No, but we can reverse engineer it.” “Cool, all we need is a food sample, a laboratory, an electron microscope.”
I was involved with a project where we had to migrate elements of a legacy “on prem” system to a new cloud-based architecture where many customer-facing elements (like which legal disclosures would display to which customer plan groups) were hard-coded in to the content management system. We knew that the disclosures had to be moved over exactly as they were, and to the same users. Yet, not a single person left at the company could verify which plans were active, which disclosures were necessary, or where the disclosures actually lived in the system because it was written in Java like 30 years ago. It would have taken the SME, who is now festooned on his nursing home bed, about 8 seconds to show us the code. Instead, we had to pay the Dick Tracy of Java developers to parachute into our system and tell us what the hell was in there. It was a nightmare for everyone except Dick Tracy, who made a fortune charging us for his services.
Let this be a lesson to all of you CTOs – you think you’re saving money by letting all of the old people go, but you’re really just passing the costs off to another business unit. Shame on you!
Conclusion
Has your project ever come to a screeching halt because an SME departed and took all of their knowledge with them? Coping with the departure of experts can be challenging. I think there are three ways to deal with this situation: talk to management, deliver iteratively, and if all else fails… reverse engineer a solution.
