Moveo.AI announced that after rigorous comparison, its custom LLM tuned for CX outperforms GPT-4-0613 in all grading dimensions, except Markdown, where GPT-4 performs better. The evaluation was based on a random sample of hundreds of entries from Moveo’s production data, which neither our LLM nor GPT-4 had encountered before. Each entry was converted into a prompt consisting of the user question, conversation history, grounding knowledge from the collection documents, live instructions, and custom instructions.
Methodology
The grading process assessed Moveo’s LLM and GPT-4 responses across 8 dimensions that capture critical traits within the CX setting:
- Hallucination
- Repetition
- Disambiguation
- Live agent handover
- Readability
- Language
- Markdown, and
- Latency
Each dimension received a score, determining which LLM provided a better response. To evaluate the performance of the different models, Moveo used a separate GPT-4 instance as a “grader,” performing a single API call for each of the samples.
Results
Moveo’s custom LLM outperforms GPT-4-0613 in all grading dimensions, except in Markdown, where GPT-4 performs better in stylistic formatting. Most importantly, it is worth mentioning that in terms of hallucination, GPT-4 performs worse, which could hurt Customer Experience. For example, if GPT-4 provides incorrect information about a product, it could lead to potential liabilities, customer dissatisfaction, and increased support requests.
Moveo’s LLM responds in only 5 seconds, while GPT-4 takes at least 18 seconds. In that time, Moveo.AI could have handled more than 4 inquiries, significantly enhancing support efficiency and customer satisfaction.
According to Panos Karagiannnis, CEO of Moveo.AI, “Enterprises need vertical-specific LLMs as every customer interaction is an opportunity to build trust and loyalty. By minimizing hallucinations and connecting to real-time information systems, our LLM significantly beats GPT-4, reduces the risk of customer dissatisfaction and potential liabilities, and sets a new standard in CX”. To learn more about Moveo’s proprietary LLMs, please visit: https://moveo.ai/
To stay updated with similar news, follow us on Martech News