Keras Turns 10: A decade of deep learning

Key Concepts

• Keras: An open-source neural network API, initially focused on ease of use and rapid prototyping, now a multi-backend library supporting TensorFlow, PyTorch, and JAX.
• Multi-Backend Support: Keras’s ability to operate with different deep learning frameworks (TensorFlow, PyTorch, JAX, etc.), providing flexibility and portability.
• Functional API: A method within Keras for defining complex models as directed acyclic graphs, offering greater flexibility than the Sequential API.
• Progressive Disclosure of Complexity: A design principle prioritizing ease of use for common tasks while allowing access to advanced customization options.
• RAG (Retrieval-Augmented Generation): A technique used in natural language processing to enhance responses by retrieving information from external sources.
• LSTM (Long Short-Term Memory): A type of recurrent neural network architecture particularly effective for processing sequential data.
• Caris Hub: A platform for sharing and accessing pre-trained models, integrated with Kaggle Models.
• Kaggle: An online community and platform for data science and machine learning competitions and collaboration.

The Evolution of Keras: A Decade of Innovation

This discussion commemorates the 10th anniversary of Keras, featuring insights from its original creators, François Chollet and Matt Watson. The conversation traces the library’s journey from a simple RNN implementation to a widely adopted, multi-backend deep learning API.

The Early Days & Founding Story (2014-2015)

François Chollet began working on the foundations of Keras in early 2014 while researching natural language processing. He developed a chatbot utilizing a rudimentary form of Retrieval-Augmented Generation (RAG) and built tooling on top of Theano, including a reusable LSTM implementation. Recognizing the need for a user-friendly deep learning API, he open-sourced the project in early 2015, initially named after the Greek word “keras” (meaning “horn”), referencing the imagery of true dream spirits in Homer’s Odyssey – representing a vision of the future realized. The initial goal was to create a deep learning library as easy to use as scikit-learn, but with greater customization options.

Keras 1.0 & Multi-Backend Support (2016)

Keras 1.0, released in March/April 2016, introduced the Functional API, significantly enhancing model-building flexibility. A pivotal moment was the release of TensorFlow in November 2015. Chollet quickly refactored Keras to support multiple backends, initially Theano and TensorFlow, recognizing TensorFlow’s potential for distributed training. By the end of 2016, over half of TensorFlow users were accessing it through Keras.

Integration with Google & Keras 2.0 (2017-2018)

Chollet joined Google in 2017, bringing Keras under the Google umbrella. However, pressure from TensorFlow leadership led to a period where Keras 2.0 became TensorFlow-centric, dropping multi-backend support. This resulted in a separation of the standalone Keras library and the TensorFlow-integrated version (tf.keras).

The Rebirth of Independent Keras & Keras 3.0 (2019-Present)

Recognizing the limitations of a single-backend approach, the Keras team reversed these changes, re-establishing Keras as an independent, multi-backend library. Keras 3.0 expanded backend support to include PyTorch, JAX, and even NumPy for inference. This independence allows users to run Keras workflows without requiring TensorFlow. The development of the core codebase for Keras 3.0 was completed in approximately two months through intensive collaborative effort.

The Role of the Community

The growth of Keras has been significantly driven by its community. Matt Watson emphasized that many of the best Keras team members originated from the user community, advocating for recruiting users directly. Community engagement is fostered through GitHub issues and pull requests, a bi-monthly online meeting, and active participation in platforms like Kaggle.

Design Principles & Usability

A core principle of Keras has always been usability. The design philosophy centers around “progressive disclosure of complexity,” ensuring simple tasks are easy while allowing advanced users to customize models extensively. The Sequential and Functional APIs have remained remarkably stable over the years, influencing API design in other libraries within the machine learning ecosystem.

Keras & Kaggle: A Synergistic Relationship

Kaggle has played a crucial role in Keras’s adoption and development. Chollet was an early Kaggle participant, and the platform helped popularize Keras. The integration of Keras Hub with Kaggle Models provides a convenient platform for sharing and accessing pre-trained models, streamlining workflows for Kaggle competitions and projects. Kaggle serves as a valuable resource for learning and evaluating machine learning techniques in a practical setting.

The Future of Keras

Looking ahead, the creators anticipate a future where Keras adapts to emerging trends like program synthesis and the fusion of deep learning with symbolic approaches. They believe the core abstractions of Keras will remain relevant, with new functionalities layered on top. They emphasize the importance of continuous learning and evaluating new technologies based on first principles, rather than solely on hype.

Notable Quotes

• François Chollet: “Simple things should be easy. Simple things should stay simple. Advanced things should be possible and the path to get there should be incremental.”
• François Chollet: “Focus on learning the concepts really well. If you find something and you can't really explain it or you don't know why, it's usually worth peeling back a few layers.”
• François Chollet: “Don't follow the hype, don't believe the hype. Whenever something new comes out, evaluate it on its own merits, from first principles.”

Technical Terms & Concepts

• Theano: An early deep learning framework, predating TensorFlow and PyTorch.
• RNN (Recurrent Neural Network): A type of neural network designed for processing sequential data.
• Distributed Training: Training a model across multiple devices or machines to accelerate the process.
• YAML: A human-readable data serialization language often used for configuration files.
• Callbacks: Functions executed at specific points during model training, allowing for customization and monitoring.
• Autodiff: Automatic differentiation, a technique used to compute gradients for training neural networks.
• MLX: A machine learning framework developed by Apple.
• OpenVINO: An open-source toolkit for optimizing and deploying AI inference.

Synthesis & Conclusion

Keras has evolved from a simple RNN implementation to a cornerstone of the modern deep learning landscape. Its success is rooted in its commitment to usability, flexibility, and a strong community. The library’s ability to adapt to new frameworks and technologies, coupled with its focus on progressive disclosure of complexity, ensures its continued relevance in a rapidly evolving field. The integration with platforms like Kaggle further solidifies Keras’s position as a valuable tool for both beginners and experienced practitioners. The future of Keras appears bright, poised to embrace emerging trends and continue empowering the next generation of machine learning engineers.