Skip to content
Neural Daily

ResearchNews

OpenAI on protecting reasoning from distillation

🇭🇺 Olvasd el magyarul →

OpenAI explained how it stops actors from training their own models on its outputs without authorization. In an internal test, extraction of GPT-6.2 Astra's hidden reasoning fell from 54% to 0.2%.

· 2 min read

Az OpenAI Combatting adversarial distillation című bejegyzésének borítóképe rózsaszín és lila háttéren
Image: OpenAI · Press kit · Source

What is the problem?

Distillation is a standard technique in which one model learns from another's outputs. In a November 20 post (OpenAI), OpenAI says the risk comes when someone uses a frontier model's outputs without authorization to train their own model, then deploys it without comparable safeguards. An attacker could, for example, distill cyber capabilities and use them in attacks the original model would have refused to help with.

Why hidden reasoning is the key

OpenAI puts special emphasis on its models' protected reasoning, the internal chain of thought users don't see. These traces carry information the final answer leaves out, which makes them especially valuable training data.

It measured this. Starting from the same pretrained student, it trained on trajectories from Astra-level teachers working on coding and defensive security tasks. Students trained on exact reasoning traces generally beat students trained only on the user-visible summaries. After RL, the exact-trace student still led at the largest inference budget tested, while the summary-trained student did better at lower budgets.

What OpenAI changed

  • At the request level, model behavior, classifiers and API controls limit reasoning extraction.
  • At the campaign level, request and account signals reveal coordinated extraction, and OpenAI enforces against the whole account network.
  • Attacks found in investigations become training and test examples, alongside automated red-teaming.
  • The safeguards now align more closely with those used for biology and cybersecurity risks.

The result in an internal adaptive evaluation: recovering protected reasoning generated for an earlier prompt succeeded 54.0% of the time on average against GPT-6 Astra with its original safeguards, and 0.2% against GPT-6.2 Astra with the updated ones (GPT-6.2 Astra and Sol: cheaper, more token-efficient).

Why it matters

For developers, reasoning on OpenAI's API is becoming more of a black box, with only summaries coming back. If you generate synthetic data to train your own models, review the terms of use, since high-volume, coordinated extraction can lead to account bans. Industry-wide, frontier labs now treat distillation as a safety issue, not just a business one.

    By Mikola Máté Patrik

    Produced with the help of AI, reviewed by an editor. How we make our articles

    Reviewed and edited by the Neural Daily desk. About

    Related articles