From Generalist To Specialist: The Role Of SFT In LLM Evolution

Olga Megorskaya is Founder & CEO of Toloka AI, a high-quality data partner for all stages of AI development.

In the race to unlock the potential of large language models (LLMs), the AI industry is no longer satisfied with LLMs that demonstrate broad knowledge. Accuracy and relevance in niche domains are the new gold standards in today’s market. LLM developers are pushing models to dominate targeted areas like coding, math, finance and other specialized fields.

In healthcare, for example, a pre-trained LLM might excel at summarizing medical journal articles but stumble when tasked with producing precise, actionable insights tailored to clinical guidelines or specific patient cases. This gap points to a challenge across industries like healthcare, law and finance: bridging the divide between general-purpose LLM capabilities and the specialized demands of real-world applications.

Supervised fine-tuning (SFT) is helping to make this connection, allowing LLMs to succeed in narrow contexts. However, to make an impact, SFT requires high-quality, domain-specific datasets. As CEO of Toloka AI, my team and I have a close-up view of how training data is collected and how it drives model improvements, as we work hand in hand with AI developers to craft expert data for SFT.

The Importance Of SFT For LLM Specialization

Training an LLM from the ground up is a resource-intensive process. As an alternative, many of our clients opt for taking a high-performance base model and using methods like retrieval-augmented generation (RAG) and SFT to adapt to their specific use case scenarios. RAG is often a faster and more cost-effective approach, as it doesn’t require large volumes of human-curated data.

However, while this method expands the model's perceived knowledge, it does not truly teach domain-specific skills. As a result, the model’s ability to analyze complex, specialized data or perform advanced reasoning within the target domain is limited. SFT can then be used to deepen the knowledge and understanding of the model in niche domains.

The SFT process is akin to mentoring a recent graduate with a solid understanding of their field who still needs real-world experience. By guiding them through scenarios and expected outcomes, a knowledgeable novice can become an expert at solving tasks within the target domain. The same applies to LLMs. When provided with properly curated data, a generic LLM can shift to a specialized model with expertise in a niche domain.

High-Quality Data: Integral To SFT

Data is the foundation of effective AI, but not just any data will do. A high-quality SFT dataset starts with relevant, unique and sufficiently complex prompts. To achieve this, human experts—experienced professionals in the targeted domains—are required to create realistic scenarios that provide context for training the LLM to respond adequately.

The backbone of a robust SFT structure is a coordinated pipeline of experts, editors and automated checks. The prompts and answers must be thoroughly evaluated before being integrated into an LLM. This involves relevancy and accuracy checks, as well as compliance with the context and guidelines.

Beyond quality assessment, it's critical to ensure the dataset fits the use case by analyzing data distribution and removing redundant or irrelevant samples. A well-curated dataset is one where each entry meets the established criteria for relevance and quality.

For example, one of our clients building a coding model collected 2,000 prompt-completion pairs per month for fine-tuning. The project focused on developing high-quality and diverse question-answer pairs and building a dataset that helps the LLM excel in Python, SQL, JavaScript and other programming languages. To achieve high technical proficiency, a network of coding experts created high-quality code snippets that accurately reflect real-world coding tasks.

A Holistic Approach To Responsible AI

While data quality is paramount, even the best data available won’t guarantee that the model behaves as expected. Developing responsible AI systems requires a holistic approach to uphold the highest standards of safety, security, impartiality and privacy.

In a holistic approach, SFT is just one step in an iterative training cycle that includes model evaluation and red-teaming or testing a model’s limitations. When weaknesses are discovered, another round of SFT is called for to address issues. Then, it’s "rinse and repeat" until the model’s output is satisfactory. This approach is a powerful way to ensure safety for critical applications.

Moving Forward: The Future Impact Of SFT

SFT’s potential goes beyond immediate business needs. It can help drive innovation in many fields by shaping LLMs that address particular challenges. In healthcare, models can be fine-tuned to assist with diagnostic processes and personalized treatment, making them more customizable and reliable. It can also be leveraged in finance to create more precise risk assessment models. Similarly, in law, complex legal information can be made more accessible by drawing clarity from nuanced legislation. These are not just technical upgrades, but a direct path to building AI that is more adaptive and trustworthy.

The true power of SFT lies in its ability to tailor AI for real-world impact. Translating the potential to reality requires a robust technological data production platform and insights from domain experts. These are the pillars upon which we can build AI that adapts and adds value where it's needed most—AI that genuinely works for everyone.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?