An AWS Machine Learning Blog post examines how to generate reasoning traces, or thinking tokens, for datasets used in supervised fine-tuning (SFT) customization that do not already include them. The authors first describe what they call the reasoning suppression problem, then present a method named Self-Distilled Reasoning (SDR). The post validates the approach across three benchmarks and offers practical recommendations, using Amazon Nova as the context for the work.
Why it matters
Many datasets used for fine-tuning lack explicit reasoning traces, which can limit a model’s ability to produce step-by-step reasoning after customization. The described SDR method is aimed at addressing that gap.
Who should care
Practitioners fine-tuning models with Amazon Nova and those working on SFT customization may find the described method and recommendations relevant.