I tested a STACK of FREE Large Language Models...here's how it went.

Key Concepts

• Large Language Models (LLMs)
• Open Source LLMs
• GPT4All
• Streamlit
• Langchain
• Prompt Templates
• LLM Chains
• Python Agent
• Model Weights
• Commercial Licensability
• Few-Shot Prompting
• Chain of Thought Reasoning

Building an LLM App with Open Source Models

Introduction

The video demonstrates how to build an application that leverages open-source large language models (LLMs) using GPT4All, Streamlit, and Langchain. The goal is to create a free, locally-run alternative to OpenAI's API and compare the performance of different open-source models against OpenAI's offerings.

Setting up the Environment

1. GPT4All GUI: The easiest way to access open-source LLMs is through GPT4All. The GUI allows you to download model weights to your local machine. The default download folder is crucial to note for later use in Langchain.
2. Streamlit App: Streamlit is used to create a simple user interface for interacting with the LLMs.
   • Import Streamlit: import streamlit as st
   • Create a title: st.title("GPT for y'all 🚀")
   • Add a text input for prompts: prompt = st.text_input("Plug in your prompt here:")
   • Implement a trigger on Enter: if prompt: st.write(prompt)

Integrating Langchain

1. Import Dependencies:
   • from langchain.llms import GPT4All
   • from langchain import PromptTemplate
   • from langchain import LLMChain
2. Define Path to Weights: Create a variable path to store the file path to the downloaded model weights (e.g., Llama 13B snoozy).
3. Create an LLM Instance: Instantiate the GPT4All class, passing the model path and setting verbose to True: llm = GPT4All(model=path, verbose=True)
4. Create a Prompt Template: Define a prompt template to structure the input to the LLM:

   prompt = PromptTemplate(
       input_variables=["question"],
       template="Question: {question}\nAnswer: Let's think step by step."
   )
   

5. Create an LLM Chain: Combine the LLM and prompt template into an LLM chain: chain = LLMChain(prompt=prompt, llm=llm)
6. Pass Prompt to LLM Chain: Update the Streamlit app to pass the user's prompt to the LLM chain and display the response:

   if prompt:
       response = chain.run(prompt)
       st.write(response)
   

Comparing Open Source Models to OpenAI

The video presents a comparison of several open-source LLMs against OpenAI's text-davinci-003 across six tests:

1. Basic Chat Q&A: Testing the model's ability to answer simple questions (e.g., difference between nuclear fusion and fission).
   • OpenAI: Coherent and accurate response.
   • MPT-instruct (7B): Decent response, but required a specific prompt template to avoid gibberish.
   • Snoozy (13B): Short, simple, and accurate response, similar to OpenAI.
2. Email Writing: Testing the model's ability to generate an email for customers about a sale.
   • OpenAI: Quick and well-structured response.
   • MPT-instruct (7B): Practical template with placeholders, but with a weird bracket and dollar sign at the beginning.
   • Snoozy (13B): Good response with a discount code.
3. Poem Writing: Testing the model's creative writing abilities.
   • OpenAI: Modern-day Shakespeare.
   • MPT-instruct (7B): Hallucinated and mistook sunflowers for "sunnies."
   • Snoozy (13B): Rhyming poem.
4. Formula One Summary: Testing the model's ability to summarize a block of text.
   • OpenAI: Neat summary.
   • MPT-instruct (7B): Failed to generate a coherent response, producing Arabic characters and hallucinating a death in the sport.
   • Snoozy (13B): Good abstractive and coherent summary.
   • Vicuna (13B): Short and sweet summary.
5. Few-Shot Prompting: Testing the model's ability to perform Chain of Thought reasoning with a sequence of numbers.
   • OpenAI: Correctly identified the condition as false.
   • Snoozy (13B): Failed to replicate the results.
   • Vicuna (13B): Chain of Thought led to the wrong conclusion.
6. Python Agent with Self-Debugging: Testing the model's ability to calculate the 12th number in a Fibonacci sequence using a Python agent.
   • OpenAI: Returned 144 (correct if starting from 1).
   • Snoozy (13B): Returned 89 (correct if starting from 0) after 17 minutes of CPU processing.

Integrating a Python Agent

1. Import Dependencies:
   • from langchain.agents.agent_toolkits import create_python_agent
   • from langchain.tools.python.tool import PythonREPLTool
2. Create a Python Agent:

   python_agent = create_python_agent(
       llm=llm,
       tool=PythonREPLTool(),
       verbose=True
   )
   

3. Replace LLM Chain with Python Agent: In the Streamlit app, replace the chain.run(prompt) call with python_agent.run(prompt).

Conclusion

The video demonstrates the feasibility of building LLM applications using open-source models. While some models may require specific prompt engineering or fine-tuning, others can achieve comparable performance to OpenAI's offerings, especially for specific tasks. The integration of a Python agent allows the LLM to execute Python code, enabling more complex tasks. The code for the application is available on GitHub.