Stanford Webinar - Ten Game-Changing Generative AI Uses for Clinical Research

Key Concepts:

• ChatGPT and other Large Language Models (LLMs)
• Critical thinking vs. rote tasks
• Data analysis and visualization
• Reproducible research
• Adversarial collaboration (red-teaming)
• Prompt engineering
• Ethical AI use (data privacy, plagiarism)
• Blinded data analysis
• Statistical analysis interpretation
• Creative thinking and brainstorming

1. Introduction of Speakers and AI's Role in Critical Thinking:

• Dr. Kristin Cobb Sainani (Stanford professor, science journalist, statistician) and Dr. Regina Nuzzo (science journalist, statistics professor at Gallaudet University and Stanford) are introduced.
• The speakers challenge the notion that AI reduces critical thinking, arguing that it enhances it by automating rote tasks, freeing up time for higher-level analysis.
• Data analysis is cited as an example: AI can handle coding, allowing more focus on data checking and bias identification.
• Kristin and Regina host the Normal Curves Podcast (normalcurves.com), which dissects scientific studies.
• A recent MIT study claiming ChatGPT fries your brain is debunked in their podcast episode, highlighting flaws in the study's methodology.

2. LLMs and ChatGPT Specifics:

• Regina emphasizes the existence of multiple LLMs (ChatGPT, Claude, Gemini), each with different strengths.
• The webinar examples use the paid version of ChatGPT, granting access to stronger models and upgraded features.
• Users are advised to disable the "improve the model for everyone" setting in ChatGPT to prevent their chats from being used for training future models, especially when dealing with sensitive research or personal health information.

3. Direct Data Analysis with ChatGPT:

• ChatGPT can perform direct data analysis by uploading datasets and asking questions, bypassing the need for coding.
• Kristin doesn't use this for her primary research yet due to the need for reproducible code and concerns about ChatGPT's potential for errors.
• Direct data analysis is useful for quick checks of open data accompanying research papers.
• Example: Analyzing a paper on flu vaccine misinformation.
  • Kristin uploaded the paper's PDF and then the publicly available Stata dataset into ChatGPT.
  • ChatGPT automatically identified variables (e.g., correction group, high concern subgroup, intention to vaccinate) without a data dictionary.
  • It generated a bar graph visualizing the distribution of vaccination intentions across groups, including sample sizes, in minutes.
  • This allowed Kristin to quickly understand the underlying data and assess the robustness of the paper's findings.

4. Adversarial Collaboration (Red-Teaming) with AI:

• AI can be used to "fight" your work, acting as a devil's advocate to identify weaknesses before others do.
• This concept is rooted in adversarial collaboration, championed by Nobel laureate Daniel Kahneman.
• AI can simulate various perspectives: reviewer number two, IRB reviewer, rival PI, skeptical journalist.
• Key to effective red-teaming:
  • Concrete Persona: Specify the perspective (e.g., "pretend you're a journalist").
  • Focused Scope: Narrow the area of criticism.
  • Appropriate Style: Define the tone (e.g., avoid snarkiness).
  • Fixed Number: Request a specific number of criticisms (e.g., "top five").
• Example: Red-teaming a podcast episode on age gaps in dating.
  • Regina uploaded the script and the research paper and asked ChatGPT to act as the study's authors.
  • ChatGPT provided five criticisms, two of which were serious and helped refine the language and anticipate potential issues.

5. Fun Uses for Life: Venting Frustrations:

• ChatGPT can serve as a virtual therapist or supportive friend for venting frustrations.
• It helps process emotions, problem-solve, and provide feedback.
• Example: Pasting a rude email into ChatGPT and asking for help responding without being mean.
• ChatGPT can even provide a "snarky friend style response" for comedic relief.

6. Fun Uses for Life: Inspiring Cartoons:

• ChatGPT can create inspiring cartoons for motivation or visualization.
• Example: Generating a cartoon of the podcast becoming successful.
• The initial result was too realistic, so Regina asked ChatGPT to make them look younger.
• Kristin uses cartoons to vent frustrations and process emotions, sometimes creating snarky cartoons she wouldn't share with the subject.

7. ChatGPT and Writing:

• ChatGPT is considered a poor writer de novo due to its training on potentially bad writing.
• It can be verbose, use passive voice, and lack focus.
• Ethical concerns exist regarding using ChatGPT to write entire essays or papers.
• However, ChatGPT can be valuable for:
  • Dictation: Transcribing spoken words into text, saving time on transcription.
    • Process: Use the microphone button in ChatGPT to record audio, then transcribe and lightly clean up the text.
    • Example: Regina and Kristin used this to create a rough script for their podcast episode on ChatGPT.
    • Kristin used it to help her son start his college essay.
  • Brainstorming: Generating ideas and exploring themes.
  • Editing: Identifying passive voice, improving conciseness, and clarifying unclear passages.

8. Blinded Data Analysis:

• Blinded data analysis prevents unconscious bias in data analysis by hiding the true data.
• Analysts work with a fake but realistic dataset where treatment labels are shuffled or numbers are jittered.
• Code is locked after analysis on the blinded data, then applied to the real data.
• Generating a good blinded dataset traditionally requires strong coding skills.
• ChatGPT can generate blinding scripts with clear comments and reproducibility in seconds.
• Example: A prompt requesting a script to add noise to a dataset and shuffle variables.
• ChatGPT provides a reproducible seed and an unblinding key.
• This lowers the barrier to entry for blinded data analysis, promoting more confident and transparent research.

9. Fun Uses for Life: Homework Help:

• ChatGPT can create practice tests for various subjects.
• Example: Generating a 40-question practice test for French 1, mimicking the level and content of the actual test.
• ChatGPT provides an answer key, saving time and effort for parents.

10. Fun Uses for Life: Health Condition Communication:

• ChatGPT can translate personal experiences and frustrations into clear clinical terms for healthcare providers.
• Example: Regina, who is deaf and uses a cochlear implant, used ChatGPT to describe her difficulty hearing birds to her audiologist.
• ChatGPT translated her descriptions into clinical terms (e.g., tweaking TNM levels and frequency).
• It also provided personalized coaching with cochlear implant-friendly cues (e.g., "metallic laser, pew, pew, pew" for a bird sound).
• This improves communication with clinicians and can lead to better outcomes.

11. Uses for Work: Translation:

• ChatGPT can translate foreign languages, medical language, legal language, and statistical analyses sections.
• Statistical analyses sections are often poorly written and difficult to understand.
• ChatGPT can translate these sections into plain English or mathematical terms.
• Example: Asking ChatGPT to interpret a statistical analysis section, specify the model, and identify the outcome variable.
• ChatGPT can identify the outcome, factors, and model, even when the authors' language is unclear.

12. Uses for Work: Kickstarting Creative Thinking (Data Visualization):

• AI can help generate data visualization ideas and code, overcoming creative blocks.
• It can provide a range of options, even if not all are perfect.
• Example: Providing ChatGPT with sensitivity and specificity data for two tasks and asking for five different visualization ideas and code.
• ChatGPT generated heat maps, confidence interval plots, and slope plots.
• AI can generate surprising and unconventional visualizations.
• The goal is not to replace human judgment but to provide a starting point for exploration and optimization.

13. Take-Home Messages:

• ChatGPT can greatly boost productivity and efficiency.
• It excels at handling rote tasks, freeing up time for critical thinking.
• Clear prompts are helpful, but iteration is often necessary.
• Skepticism is crucial: ChatGPT makes mistakes and hallucinates.
• It's a terrible original writer but a good editor.
• It's valuable for both work and everyday life.

14. Q&A Highlights:

• The speakers used the paid version of ChatGPT (ChatGPT 4 and 5).
• Ethical guidelines for using ChatGPT:
  • Avoid uploading non-anonymized data due to HIPAA rules.
  • Use anonymized or de-identified data.
  • Blind the data set before uploading.
  • Use ChatGPT to write code to de-identify data.
• Distinguishing between augmenting writing and unfair use:
  • ChatGPT is a bad writer and should not be used to write entire essays or papers.
  • It's helpful for brainstorming, outlining, and editing.
  • The final product should be in your own words.
  • It can be used for administrative writing tasks.
• Future of AI in research:
  • Integrating ChatGPT into peer review to identify statistical errors.
  • Using AI to create guardrails for AI.
  • Developing better methods for teaching ChatGPT and critical thinking skills simultaneously.

Synthesis/Conclusion:

The webinar provides a comprehensive overview of how ChatGPT and other LLMs can be used to enhance productivity, creativity, and critical thinking in various domains, from data analysis and writing to personal tasks and health communication. While emphasizing the importance of ethical considerations, data privacy, and the need for human oversight, the speakers demonstrate the potential of AI to automate rote tasks, unlock new insights, and empower individuals to work more efficiently and effectively. The key takeaway is that AI should be viewed as a tool to augment human capabilities, not replace them, and that critical thinking remains paramount in the age of AI.