Introducing Flax NNX (Part 2)

Key Concepts

• NNX Module: Fundamental building block in NNX, providing a class-based way to define networks.
• Python Graphs: Mechanism allowing standard Python objects and semantics (mutability, reference sharing) to be compatible with Jax.
• NNX Param: Manages state explicitly in NNX.
• Functional API: Separates module structure from its state.
• JIT (Just-In-Time Compilation): Compiles Python code to optimized machine code using XLA for specific hardware.
• Pure Function: A function that always returns the same output for the same input and has no side effects.
• Side Effects: Actions that interact with the wider world or program state outside the function (e.g., modifying global variables, print statements, file I/O).
• Jax Transformations: Functions like jax.jit, jax.grad, and jax.vmap that operate on pure functions and pi trees.
• Pi Trees: Data structures used by Jax transformations.
• NNX Transformations: Wrappers around Jax transformations (e.g., nnx.jit, nnx.grad, nnx.vmap) designed to work with stateful NNX objects, automatically managing state.
• XLA (Accelerated Linear Algebra): A compiler for linear algebra that can optimize Jax code for different hardware backends.

Performance and NNX.jit

• NNX.jit is presented as a "supercharger" for NNX code, analogous to torch.compile in PyTorch but potentially faster.
• JIT compilation analyzes Python code and compiles it to highly optimized machine code using XLA, targeting specific hardware.
• The primary requirement for effective JIT usage is that the function being jitted should be a pure function.

Performance Comparison: JIT vs. Non-JIT

• A simple loop example is used to demonstrate the performance benefits of NNX.jit.
• PyTorch (non-jitted): 79.9 milliseconds per loop.
• NNX.jit (first run, compilation time included): 79.4 microseconds per loop.
• NNX.jit (second run, cached compilation): 39.9 microseconds per loop.
• This demonstrates a speedup of approximately 2000x (specifically, 1989x) compared to PyTorch.
• A pure Jax version is even faster than the NNX version.

Pure Functions: Definition and Importance

• A pure function behaves like a mathematical function: same inputs always yield the same output.
• Critically, a pure function does nothing else besides compute that output; it has no side effects.
• The golden rule: only apply jax.jit or nnx.jit to pure functions.
• Aiming for pure functions is good coding practice in general, as they are easier to understand, test, and debug.

Side Effects: Examples and Implications

• Side effects are actions that reach outside the function to interact with the wider world or program state.
• Examples:
  • Modifying a global variable.
  • Appending an item to a list passed into the function (modifying the original list).
  • Print statements.
  • Reading files.
  • Interacting with networks.

Handling Python Control Flow with JIT

• Problem: Standard Python control flow (if statements, while loops) that depends on the value of an input argument can cause issues with JIT.
• Jax traces the function once based on the shapes and types of the inputs, not the values.
• If the code path taken depends on the value of an input during tracing, the compiled code may be incorrect for different input values, leading to errors or recompilations.
• Solution:
  1. Identify the pure calculations inside the if/else blocks or loop body.
  2. Put those calculations into their own separate pure functions.
  3. Apply nnx.jit to those smaller functions.
  4. Keep the Python if/while logic in the outer function (which is not jitted) and have it call the smaller jitted functions as needed.

NNX Transformations vs. Jax Transformations

• Jax transformations (e.g., jax.jit, jax.grad, jax.vmap) are designed for pure functions and work with pi trees.
• NNX modules hold their own state (parameters, optimizer state, RNG keys), making them stateful.
• NNX transformations bridge the gap between Jax's pure function requirement and NNX's stateful modules.
• NNX transformations are wrappers around Jax transformations that automatically handle state management for NNX objects.
• When using nnx.grad or nnx.jit on NNX objects, the NNX transformation layer automatically figures out the involved state, splits the object into state and structure, runs the transformation, and merges the updated state back into the NNX objects.

When to Use NNX Transformations vs. Jax Transformations

• NNX Transformations: Use when working with NNX objects (NNX modules, optimizers, state variables like NNX.param, NNX.RNGs). They simplify state management.
• Jax Transformations: Use for functions that are naturally pure and don't involve NNX objects (e.g., data loading, pre-processing functions operating on Jax arrays). Also useful for low-level control or niche Jax transformations without NNX equivalents.
• Jax transformations can be faster (e.g., jax.jit can be twice as fast as nnx.jit), but NNX transformations are recommended for building and training models with Flax NNX due to their state management capabilities.

Conclusion

The video emphasizes the importance of understanding pure functions and side effects when using JIT compilation in Jax and NNX. It highlights how NNX transformations simplify the process of working with stateful NNX modules by automatically managing state during JIT compilation and other transformations. The key takeaway is to use NNX transformations when working with NNX objects to streamline the development process and leverage the performance benefits of JIT compilation effectively.