Cách dùng model AI gpt-oss-20b miễn phí trên Mac và Win, dễ lắm ai cũng làm được

Key Concepts

• Open weight model
• GPT OSS 20B (20 tỷ tham số)
• GPT OSS 120B (120 tỷ tham số)
• LM Studio
• Reasoning effort (Low, Medium, High)
• Context length
• GPU offload
• System prompt

Main Topics and Key Points

Introduction to Open AI's Open Weight Models

• Open AI has released two open weight models: GPT OSS 20B and GPT OSS 120B.
• The term "open weight" is used instead of "open source."
• GPT OSS 20B can run on personal devices with around 16 GB of RAM.
• GPT OSS 120B is larger, more intelligent, and more accurate but requires more powerful hardware.
• The video will guide users on how to run these models on their computers and share initial experiences using them for data-related tasks, specifically SQL coding.

Reasons to Use These Models

• The models are suitable for common coding and data tasks.
• They can run directly on a computer, eliminating the need for cloud services or subscriptions.
• Usage is limited only by the computer's hardware capabilities.
• The models can be used offline without an internet connection.
• Open AI's release of an open weight model is significant, as most people are familiar with their closed models like GPT-4O.
• Users can fine-tune and customize these models.

Expectations and Limitations

• The 20B parameter model is relatively small and may not be suitable for all tasks.
• Even the 120B parameter model has limitations.
• Users are encouraged to experiment to see if the models fit their needs.

Using LM Studio to Run the Models

• LM Studio is a software used to run the models on computers.
• It supports macOS, Windows, and potentially Linux.
• RAM 16 GB is sufficient for macOS.
• Windows users can choose between CPU or dedicated GPU processing.
• Users can download LM Studio from its website and install it.
• LM Studio supports various models from different providers, including Google and Microsoft.
• Users can search for "GPT OSS" within LM Studio to find the two models.
• The 20B parameter model is suitable for most machines, while the 120B parameter model requires more powerful hardware (e.g., 128 GB RAM).

Running and Testing the Models

• After downloading a model, users can load it within LM Studio.
• The models can answer questions in multiple languages, including Vietnamese and English.
• The "reasoning effort" setting allows users to control the depth of the model's reasoning process (Low, Medium, High).
• Higher reasoning effort leads to longer processing times and more token usage.
• The 20B parameter model provides good reasoning and content generation in Vietnamese.

Real-World Applications and Examples

• The presenter used the 20B parameter model to generate SQL code and modify existing SQL frameworks.
• The model accurately generated SQL queries based on specific instructions.
• The presenter found the 20B model sufficient for their SQL-related tasks, similar to using ChatGPT.
• The model can handle iterative question-and-answer sessions, allowing for fine-tuning of the results.
• The model understands and works well with BigQuery SQL dialect.

Advanced Settings and Customization

• Context Length: Adjusts the length of the conversation history the model considers.
• GPU Offload: Allows users to offload processing to the GPU.
  • If the GPU has sufficient VRAM, users should maximize GPU offload.
  • If the GPU is not powerful enough, it's better to use CPU processing and set GPU offload to zero.
• System Prompt: Allows users to define the model's behavior and response style.
  • Examples: "Always answer in Vietnamese," "Keep replies short and clear," "Prioritize bullet-point answers for detailed explanations."

Benefits of Open Weight Models

• The models run locally, ensuring data security and privacy.
• Users don't have to worry about violating data security policies.
• The presenter recommends consulting with the company's IT department before using these tools.
• The release of open weight models is seen as a positive development, potentially leading to fine-tuned models for specific tasks (e.g., coding, novel writing, Southeast Asian languages).

Conclusion

• LM Studio is easy to use, and the GPT OSS models are effective.
• The 20B parameter model is sufficient for SQL-related tasks.
• Further testing and experimentation are needed to fully evaluate the models' capabilities.
• The presenter encourages viewers to share their experiences with the models.
• Future videos will test the models on various machines, including macOS and Windows laptops with different hardware configurations.

Notable Quotes

• "Với những cái tác vụ khác á thì các bạn cứ thử đi tại vì free mà coi là nó có thể đáp ứng được tới mức nào." (For other tasks, just try it out because it's free to see how well it can handle them.)
• "Mình khá là tin vào cái khả năng liên quan tới code lập trình và xử lý data của con GPT ha. Tại vì ch GPT đã làm việc đó rất là tốt." (I'm quite confident in the coding, programming, and data processing capabilities of the GPT model. Because ChatGPT has done that very well.)
• "Đó thì cái phần system prom là một cái rất là hiệu quả để mà các bạn có thể chỉnh được cái câu trả lời nó sát với lại ý muốn của các bạn hơn." (So the system prompt is a very effective way for you to adjust the answer to be closer to what you want.)

Technical Terms and Concepts

• Open Weight Model: A model where the weights (parameters) are publicly available, allowing for local execution and customization.
• Parameters: Variables within a machine learning model that are adjusted during training to improve performance. More parameters generally mean a larger and more capable model.
• LM Studio: A software application that allows users to run large language models locally on their computers.
• Reasoning Effort: A setting that controls the depth and complexity of the model's reasoning process.
• Context Length: The amount of text or conversation history the model considers when generating a response.
• GPU Offload: The process of transferring computational tasks from the CPU to the GPU to accelerate processing.
• System Prompt: A set of instructions or guidelines that define the model's behavior, tone, and response style.
• Fine-tuning: The process of further training a pre-trained model on a specific dataset to improve its performance on a particular task.

Logical Connections

The video begins by introducing the new open weight models from Open AI and then explains why users should consider using them. It then provides a step-by-step guide on how to set up and run these models using LM Studio. The presenter shares their initial experiences using the models for SQL-related tasks, highlighting their strengths and limitations. Finally, the video explores advanced settings and customization options within LM Studio, emphasizing the benefits of open weight models for data security and privacy.

Data, Research Findings, or Statistics

• GPT OSS 20B requires approximately 16 GB of RAM to run.
• GPT OSS 120B requires significantly more RAM, with the model file itself being over 60 GB and requiring around 80 GB of RAM to load.
• The presenter's MacBook Pro M4 Max with 128 GB of RAM can run the 120B parameter model with full GPU offload.

Synthesis/Conclusion

The video provides a practical guide to using Open AI's new open weight models, GPT OSS 20B and GPT OSS 120B, with LM Studio. It highlights the benefits of these models, including local execution, data security, and customization options. The presenter's initial experiences suggest that the 20B parameter model is suitable for common coding and data tasks, particularly SQL-related work. The video encourages users to experiment with these models and share their experiences, contributing to the growing community around open weight AI.