The New Code — Sean Grove, OpenAI

Key Concepts

• Specifications: Written documents that clearly and unambiguously express intentions and values, serving as a source of truth for aligning humans and AI models.
• Code vs. Communication: The argument that structured communication is the primary bottleneck in software development, with code being a secondary artifact.
• Vibe Coding: A development approach where communication of intent precedes code generation, often using prompts with AI models.
• Model Spec: An example of a specification, specifically OpenAI's document outlining the intended behavior and values of their models.
• Deliberative Alignment: A technique for aligning AI models with specifications by using a grader model to score responses against the spec and reinforce aligned behavior.
• Syphancy: The tendency of AI models to be overly flattering or ingratiating, often at the expense of truthfulness.

Code vs. Communication

• The speaker argues that while code is often seen as the primary output of a programmer's work, it only represents 10-20% of the actual value they provide.
• The remaining 80-90% lies in structured communication, encompassing activities like:
  • Understanding user challenges
  • Distilling user stories
  • Ideating solutions
  • Planning implementation
  • Sharing plans with colleagues
  • Translating plans into code
  • Testing and verifying the impact of the code
• Structured communication is identified as the bottleneck in software development, encompassing:
  • Knowing what to build
  • Knowing how to build it
  • Knowing why to build it
  • Knowing if it has been built correctly and achieved its intended goals
• The speaker posits that as AI models become more advanced, the ability to communicate effectively will become the most valuable programming skill.

Vibe Coding and the Importance of Specifications

• Vibe coding is presented as an example where communication precedes code, with the model handling the "grunt work."
• However, the current practice of discarding prompts after generating code is criticized as analogous to "shredding the source and version controlling the binary."
• The speaker emphasizes the importance of capturing intent and values in a written specification, which serves as the source of truth.
• A written specification enables:
  • Alignment of humans on shared goals
  • Synchronization on what needs to be done
  • Discussion, debate, and reference
• Without a specification, only a "vague idea" exists.

Specifications as a More Powerful Alternative to Code

• Code is described as a "lossy projection" from the specification, similar to decompiling a binary and losing comments and variable names.
• Code often doesn't embody all the intentions and values, requiring inference to understand the ultimate goal.
• A written specification encodes all necessary requirements for generating code.
• A robust specification can be translated to multiple target architectures (e.g., TypeScript, Rust, documentation, tutorials).
• The speaker challenges the audience to consider if their codebase could be used to generate a compelling podcast that teaches users how to succeed, implying that much of the valuable information resides outside the code itself.
• The new scarce skill is writing specifications that fully capture intent and values.

Anatomy of a Specification: The OpenAI Model Spec

• The OpenAI model spec is presented as a living document that expresses the intentions and values OpenAI hopes to imbue its models with.
• It is open-sourced and implemented as a collection of markdown files on GitHub.
• Markdown is chosen for its human readability, version control capabilities, and accessibility to non-technical contributors (product, legal, safety, research, policy).
• Each clause in the model spec has a unique ID (e.g., sy73).
• For each clause, a corresponding markdown file (e.g., sy73.md) contains challenging prompts that serve as success criteria for the model.

Case Study: The 40 Syphancy Issue

• The speaker discusses an incident where OpenAI models exhibited excessive syphancy (flattery), which eroded trust.
• The model spec includes a section dedicated to avoiding syphancy, explaining that it is harmful in the long term.
• The existence of this specification allowed OpenAI to:
  • Align humans around the value of avoiding syphancy
  • Identify the behavior as a bug
  • Roll back the model
  • Publish studies and blog posts
  • Fix the issue
• The spec served as a "trust anchor" during the incident, communicating expected and unexpected behaviors.

Making Specifications Executable and Aligning Models

• The speaker introduces a technique called "deliberative alignment" for automatically aligning models with specifications.
• The process involves:
  1. Taking the specification and challenging input prompts.
  2. Sampling responses from the model under test.
  3. Giving the prompt, response, and policy to a grader model.
  4. Asking the grader model to score the response according to the specification.
  5. Reinforcing the model's weights based on the score.
• This technique moves policy enforcement from inference time to the model's weights, allowing the model to "muscle memory" the policy.
• Specifications can encompass various aspects, including code style, testing requirements, and safety requirements.

Specifications as Code

• Even though the model spec is just markdown, it is useful to think of it as code.
• Specifications are:
  • Composable
  • Executable
  • Testable
  • Shippable as modules
• Similar to programming, spec authorship benefits from tools like:
  • Type checkers (ensuring consistency between specifications)
  • Linters (identifying ambiguous language)
• Specs provide a toolchain targeted at intentions rather than syntax.

Lawmakers as Programmers

• The US Constitution is presented as a national model specification.
• It includes:
  • Written text that serves as clear policy
  • A versioned way to make amendments
  • Judicial review (a grader assessing alignment with policy)
  • Precedents (input-output pairs that disambiguate and reinforce the policy)
  • Chain of command
• The enforcement of the Constitution over time is a training loop that aligns citizens towards shared intentions and values.
• The speaker suggests that lawmakers may become programmers, or vice versa.

Universal Application of Specifications

• Programmers align silicon via code specifications.
• Product managers align teams via product specifications.
• Lawmakers align humans via legal specifications.
• Prompt engineering is a form of proto-specification, aligning AI models towards common intentions and values.
• Anyone writing prompts is a spec author.
• Specs enable faster and safer shipping, and allow for broader contribution.

Conclusion and Call to Action

• Software engineering has always been about solving human problems, not just writing code.
• The industry is moving from disparate machine encodings to a unified human encoding of solutions.
• The speaker encourages the audience to:
  • Start with a specification for their next AI feature.
  • Debate the clarity and communication of the spec.
  • Make the spec executable.
  • Test the model against the spec.
• The speaker poses the question of what the IDE (integrated development environment) of the future will look like, suggesting it might be an "integrated thought clarifier."
• The speaker concludes with a request for help in aligning agents at scale, inviting the audience to join the new agent robustness team and contribute to delivering safe AGI.