When I asked ChatGPT to write this piece for me it responded, in a Bristolian accent: “Not another one?”
If even AI is getting bored generating self-referencing material, care is needed in adding to the plethora of recent comments. Hopefully what follows goes beyond what you’ve already heard to explore some practical implications of how generative AI might work in our highly regulated sphere (and where it might fail) and especially how it might help with the communication elements of the FCA’s Consumer Duty.
For a very modest fee we can all use a Large Language Model (LLM) to answer any question that comes to mind. The output maybe sufficient to satisfy our curiosity, get that tricky speech written or even to pass an exam but are they of a standard that would pass a Compliance audit or Regulatory inspection? How accurate do answers need to be?
LLMs can trawl the internet for all the accumulated wisdom and knowledge available but, aside from creating accidental mistakes or “hallucinations”, trawling the web means encountering factual as well as the opinion driven content: consider the question “Is it better to save in an ISA or a Pension?” There is no definitive answer but some strong pointers – like a matching employer’s contribution or a need to access the proceeds before age 55 – but what do you do with the blog post or national newspaper article that suggest that “buy to let” is the real answer?
One response is to severely restrict the trawl to a well curated library: the sources need to be varied, factually accurate but also UpToDate. Equally if the sources are few in numbers there is less scope to generate anything new. The recent abolition of the Lifetime Allowance and maximum annual contributions in pensions is a good example: the LLM not only needs to know this has happened but references to the previous state of affairs need to be relegated to history in the whole library to ensure the output is appropriate.
But what sort of output are we talking about: a page for the organisation’s website, marketing material or perhaps the answer to a specific customer query? Just as happens today, the first draft can be given to others to edit for style and accuracy if the piece is to be widely disseminated.
Alternatively, automated assurance models – a second use of LLM that edits the output of the first LLM – might be developed. The library used by the second may well be smaller than the first: it’s about verifying the facts, not generating new content. An automated assurance model may have limited advantages in terms of cost or accuracy over the human editor for widely used content that needs to be as close to 100% accurate as possible: fewer copywriters but otherwise much the same as today?
When it comes to answering customer queries, the cost of human involvement is much more of an issue as demonstrated by the recent history of centralised call centres, followed by offshoring, automated chatbots and the absence of affordable financial advice or guidance to any bar the most well-off quartile. One can prepare an FAQ but, when dealing with individuals, responses need to be personalised and timely. The answers also need to be accurate, albeit it’s fair to say there is more leeway with a one-off comment than a prominent webpage. Whether by phone or in a written exchange, assurance today is provided by training staff beforehand and monitoring them after the event. An automated assurance model might be able to do this in real time at a much lower cost.
Dealing with individual customer queries thus seems to be the area where LLMs will have the most impact on how customers are served. Indeed, with implementation of the FCA’s Consumer Duty imminent, Firms are expected to:
“…communicate and engage with customers so that they can make effective, timely and properly informed decisions about financial products and services and can take responsibility for their actions and decisions”. (FCA FG 22-5 Section 1.9)
If a provider has a large existing client bank, satisfying their Consumer Duty obligations with humans will be a very expensive process – whether it’s getting behind with mortgage payments in a time of higher interest rates or understanding the impact of a withdrawal from one’s pension in terms of income tax to be paid or even the loss of means-tested benefits, there simply aren’t the people available to answer at a salary/cost that is acceptable to both the provider and the customer.
On the other hand, consider an Artificial Intelligence system that provided an accurate answer to the consequences of going further into arrears or how much extra tax would be payable if one withdrew all of one’s pension. This part needs to be absolutely accurate. The LLM could then take part in a discussion about other options available to solve the underlying problem. This doesn’t have to be exhaustive and, as is the case with any discussion, it may be less amenable to an exact measure of accuracy. Nevertheless, it could be invaluable for a consumer in making an informed decision. Further there would probably still be access to a human to “close the deal” (but not to reopen the whole discussion). Such hybrid use of automated and human guidance could keep costs to levels providers can afford while providing guidance to consumers that is simply unavailable today.
The implications of a repeatable process and a clearly defined library of source material facilitate Compliance audits in a way unimaginable in the past. Of course, should systemic errors occur, they will inevitably be all the more expensive to resolve. Then again, Brenda’s famous meme could have been used in the context of financial services mis-selling/advice scandals in pretty much every year of the last 30 or so, all before LLMs had been dreamt of and any automation was conceivable.