Cerebras DocChat Launched: Built on Llama 3, DocChat Delivers GPT-4-Level Conversational Quality Control Trained in Hours

The release of DocChat by Cerebras marks a major milestone in document-based conversational question-answering systems. Cerebras, known for its deep expertise in machine learning (ML) and large language models (LLM), has introduced two new models to the DocChat series: Llama3-DocChat Brains And Dragon-DocChat BrainsThese models are designed to deliver high-performance conversational AI, specifically tailored to document-based question answering tasks, and were developed at unprecedented speed using Cerebras’ cutting-edge technology.

DocChat Templates Overview

Cerebras Llama3-DocChat is based on Llama 3 and incorporates advanced insights from recent research in the field, including Nvidia’s ChatQA model suite. Developing this model required leveraging extensive experience in LLM training and dataset curation, as well as innovative techniques such as synthetic data generation. This approach allowed Cerebras to address limitations that could not be fully addressed using available real-world data.

Cerebras Dragon-DocChat is a multi-round retrieval model optimized to improve recall rates. The model was trained on the ChatQA conversational question-answering dataset and enhanced using a contrastive loss with hard negatives, leading to significant improvements in recall rates over its predecessors and competitors.

Training effectiveness and performance

One of the most remarkable features of the DocChat models is the speed at which they were trained. The Cerebras Llama3-DocChat model was trained in just a few hours using a single Cerebras system, while the Dragon-DocChat model was fine-tuned in minutes. This remarkable efficiency is a testament to Cerebras’ advanced hardware and software capabilities, setting a new benchmark in the AI ​​industry.

The performance of these models was rigorously evaluated in multiple benchmarks. Both models achieved top-notch results for their respective sizes, outperforming many existing solutions. For example, on benchmarks like ConvFinQA and SQA, Cerebras Llama3-DocChat showed significant improvements, demonstrating its superior ability to handle complex conversational question-answering tasks.

Open Source Commitment

Cerebras also reaffirmed its commitment to the open source community by releasing DocChat. The company made model weights, full training recipes, and associated datasets publicly available. This level of transparency allows other AI researchers and developers to replicate, build upon, and innovate on Cerebras’ work, potentially leading to new advances in the field.

Reference comparisons

Cerebras DocChat models have shown impressive results in head-to-head comparisons with other models. For example, in the ChatRAG benchmark, Cerebras Llama3-DocChat outperformed Nvidia Llama3-ChatQA and GPT-4 Turbo in several key metrics. Similarly, Cerebras Dragon-DocChat outperformed Facebook’s Dragon+ and Nvidia’s Dragon Multiturn in terms of recall rates, especially in multi-turn conversation contexts.

Developing DocChat has been a challenge. One of the key issues addressed during training was the model’s ability to handle unanswered questions. Early testing showed that the model struggled to answer these questions, often failing to respond appropriately. Through experimentation, Cerebras found that oversampling the samples corresponding to unanswered questions improved the model’s performance. However, the company acknowledges that there is still room for improvement in this area, especially when compared to state-of-the-art models like QuAC and DoQA.

Another challenge was improving the model’s arithmetic performance, which was initially prone to errors. By incorporating techniques inspired by the Chain of Thought (CoT) method, Cerebras significantly improved the model’s accuracy on arithmetic tasks. Feature extraction was challenging due to the need for higher-quality training data. This problem was alleviated by incorporating a subset of SKGInstruct, an instruction tuning dataset that improved the model’s performance on feature extraction tasks.

Cerebras has ambitious plans for the future development of the DocChat series. The company is exploring several interesting avenues, including support for longer contexts, improved mathematical reasoning, and larger model sizes. These improvements should further solidify Cerebras’ position as a leader in conversational AI.

In conclusion, Cerebras’ release of DocChat, the speed and efficiency with which these models were trained, and their top-notch performance underscore Cerebras’ technological prowess. Additionally, the company’s commitment to open source and continued innovation ensures that DocChat will benefit its users and contribute to the broader AI community. As Cerebras continues to refine and expand its offerings, DocChat’s impact on the future of AI-driven communication will likely be profound.

---

Discover the Model on HF and details. All the credit for this research goes to the researchers of this project. Don’t forget to follow us on Twitter and join our Telegram Channel And LinkedIn Groops. If you like our work, you’ll love our bulletin..

Don’t forget to join us Over 49,000 ML subreddits

Find the next webinars on AI here