### Scale’s Paper Overview

Scale’s paper, _Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs_ ( [arxiv](https://arxiv.org/pdf/2409.19759)), has been accepted to the [Fine-Tuning in Machine Learning (FITML) Workshop at NeurIPS 2024](https://sites.google.com/view/neurips2024-ftw). This post overviews the research presented in that paper.

**Paper Authors:** Yung-Chieh Chan, George Pu, Apaar Shanker, Parth Suresh, Penn Jenks, John Heyer, Sam Denton

In this paper, the Scale AI team investigates various synthetic data generation strategies for fine-tuning LLMs under different constraints, paying particular attention to cost-effectiveness. This makes this paper especially relevant for real-world enterprise fine-tuning considerations.

### Data Generation Strategies

Fine-tuning requires two key steps: creating a high-quality dataset and fine-tuning the model to acquire new capabilities. Various solutions have emerged to overcome the data bottleneck, including:
- Manual or automated enhancement of data quality
- Increasing the dataset quantity
- Extracting more informative learning signals from each data sample

While each of these methods has shown promise, their relative cost-effectiveness and performance across different tasks and data constraints remains unclear, especially in resource-constrained scenarios.

### How We Did It:

At Scale, we perform various experiments to better understand the data layer while building custom LLMs for various domains and tasks. Recently, we have investigated this question using tools from [Scale GenAI Platform](https://scale.com/genai-platform) to perform a controlled experiment grouping various synthetic data strategies.

We train our models with Supervised Fine-Tuning (SFT), which involves a dataset of prompts and responses, and consider three main synthetic data strategies: _Answer Augmentation_, _Question Rephrase_, and _New Question_. Each strategy represents a different way of modifying the prompt or response space.

### What We Found:

We show which methods maximize accuracy given a different number of starting prompts and a different “budget” of synthetic data. We measure the "query budget ratio" — a ratio specified by q/s, where "q" is the query budget (number of calls to an LLM) and "s" is the initial seed instruction size.

Surprisingly, this "shift" in the optimal method is consistent across the three tasks. With a limited budget, creating new responses is most effective. As we increase our budget, creating new prompts yields the most effective results.

We investigate important factors affecting performance under constrained budgets, revealing that the choice of augmentation strategy is less critical in data-rich scenarios.

### Conclusion

Our work addresses the fine-tuning data bottleneck challenge by offering a framework that guides enterprises in defining and refining their synthetic data generation strategies to maximize cost-efficiency within budget constraints. Our findings reveal that the optimal strategy hinges on the query budget ratio, and that verification of synthetic responses has less impact on performance.