DRY – Use an Insight Engine
Over the Christmas holiday, I set out to develop a prototype for an ITSM/Customer Support solution. In this solution, I wanted to assist two primary use cases. The first was to enable an end customer with self-support. The second was to enable the agent with the necessary information to be more proficient. In both cases I wanted the solution to be aided by Machine Learning and Natural Language Processing (i.e. an Insight Engine).
As a customer, I want self-service
Many of us may have experienced this scenario over the holidays. You buy something from BestBuy or receive a gift. Then there some minor issues with the product so you goto the manufactures website and its abysmal. The manufacturer has been acquired and you are simply unable to find drivers or the missing information to configure the remote. Thank god they have a chatbot, but not. (FWIW, chatbots generally suck but tell that to the Director of Marketing or Customer Support)
As an agent, I want recommended articles to be presented
Similarly, when working with a customer service agent, how many times are you placed on hold while they look up your information or attempt to take you a support process. Agents are under intense pressure to resolve the case quickly, are required to typically access multiple data sources, construct advanced queries in specific ways in order to be effective, and adapt to ever-changing consumer expressions. Ideally, the practices of the best agents need to be mimicked by the entire organization.
What would the system look like?
This is something that could benefit from machine learning. With machine learning we could:
- Predict the best articles that would help solve the case for the agent
- Suggest solutions to the customer that would deflect the creation of the case
- Monitor the system to detect anomalies (i.e. a spike in a particular type of issues)
- Understand the characteristics of the case to correctly assign it to the right queue
- Train the system on the characteristics of successful outcomes to predict future interactions
Given this problem and desired functionality, how would we go about solving it? We would start by ingesting the various collections of information such as:
- The ticketing system for customers
- The bug tracking system from engineering
- The internal knowledge base
- The external knowledge base
- The previous knowledge base of the acquired company
This would be done through an ETL (extract, transform, and load) process which we would gather up all of the documents of interest, we would gather up the security entitlements (we don’t want customers seeing things that are meant for internal users), and we would want updates as they happen in near real-time. We need to do this so we can process them with our machine learning algorithms on an ad hoc basis.
A key part of that L process is loading it into some data store. Given we want to do intense processing on random data, where should we store this data? We could look at a NoSQL store that allows easy retrieval and scale. But for our classification and language requirements, it would make more sense to use an inverted index. Lucene has become the defector baseline technology in this area. (i.e. an Inverted Index)
Finally, we would need machine learning processes and a means to present this information in some logical manner.
How an Insight Engine Accelerates Development
I don’t have a lot of time and my ADHD prevents me from caring too much on some of these low-level details. If only there was some technology stack that already existed so I could adhere to the DRY principle. If this unicorn did exist, I could focus solely on pieces that are specific to my solution. Luckily for us and my attention, there is. It’s called an “Insight Engine”. An Insight Engine is more than a “search++ engine”. An Insight Engine is the evolvement of search from classic retrieval to machine learning platform has created a new class of system. Gartner has created a scoring for this technology in their “Gartner Magic Quadrant for Insight Engine”. My company, MC+A was actually listed in it this year through one of our partners so I’m slightly biased.
With an Insight Engine running, I have a platform that I can ingest from a variety of data sources. In that process, I can manipulate the data prior to indexing and use it’s out of the box algorithms to provide those “Next Best Actions”. Alternatively, I could roll my own by it seems way more complicated.
I often wonder why businesses roll there own. You’d be spending time and money to simply recreate what is already out there. Connectors are the plumbing that no one wants to maintain for example. So I’m often puzzled what makes a company use Adobe Experience Manager for their website but then tries to roll their own machine learning/insight platform. Either way, stay tuned. Looking forward to sharing the ITSM solution that I’ve been working with.