Education

Computational lab course that spans research data processing workflow starting just after the point of acquisition through to computation and visualization. Topics will span Stanford specific best practices for data storage, code management, file formats, data curation, toolchain creation, interactive and batch computing, dynamic visualization, and distributed computing. Students will work with datasets of their choosing when working through topics. Course information at: http://bioe301p.stanford.edu

This one-week course introduces core tools for research computing through short lectures and guided, hands-on coding. Students will learn to use command line Bash and work effectively in the Unix shell; track, version, and share code with Git; transfer research data between Stanford data storage systems; access Stanford HPC resources (FarmShare, Sherlock) via the browser-based Open OnDemand portal; and use Python for scientific computing and reproducible analysis. Prerequisite: knowledge of basic Python syntax and data types.

Building on Data Best Practices: Basics, this one-week course covers advanced topics in reproducible, portable, and scalable scientific computing. Students will containerize software and environments with Podman/Docker; parallelize analyses locally with GNU Parallel and orchestrate cloud-based, asynchronous parallelization with Google Cloud Pub/Sub; and implement CI/CD in Stanford GitLab, using runners to automate scientific data preprocessing and analysis pipelines. It is recommended that students take Data Best Practices: Basics prior to enrolling in this course. Students with equivalent experience may also enroll.

For first-year Neurosciences IDP graduate students. This hands-on course introduces core tools for research computing. Students will learn to use command line Bash and work effectively in the Unix shell; track, version, and share code with Git; transfer research data between Stanford data storage systems; access Stanford HPC resources (FarmShare, Sherlock) via the browser-based Open OnDemand portal; and use Python for scientific computing and reproducible analysis. Students will also be introduced to popular, open-source neuroscience-specific Python packages.

Please log in to view the schedule and apply for workshops.

This workshop will introduce attendees to command line scripting in Linux environments, which are the backbone of high-performance computing and AI. Additionally, this workshop is recommended for anyone who is broadly interested in the Data Best Practices workshop series, as this skill is essential for participating in more advanced workshops.

Topics Covered

  • Introduction to Unix shells
  • Essential Bash commands
  • Developing and executing a Bash script

This workshop introduces version control as a fundamental tool for reproducible research. Attendees will gain hands-on experience using Git and Stanford GitLab to track changes to code, collaborate effectively, and help ensure reproducible scientific computing.

Topics Covered

  • Introduction to version control concepts
  • Common git commands and workflows
  • Getting started with Stanford GitLab

This workshop provides a foundation in Python specifically tailored for scientific computing. You will learn to use essential libraries like NumPy and Pandas for data manipulation and Matplotlib/Plotly for visualization.

Topics Covered

  • Basic syntax, data types, and core libraries
  • Jupyter
  • NumPy
  • Pandas

This workshop provides a practical introduction to command line agents, which are AI tools that read, write, and run code directly in your terminal. You will install and configure an agent, then complete tasks that each demonstrate a reusable capability: directing an agent with written specification files, delegating tools you have not learned, connecting to external APIs, processing documents at scale, and retrieving and visualizing public data. Every exercise in the workshop produces output that is directly verifiable against its source, which is essential when working with agents.

Topics Covered

  • Claude Code
  • Codex
  • Gemini CLI

This workshop is a great opportunity to learn about ways to manage large volumes of data without unlimited Stanford Google Drive. You will also gain hands on experience with command line data management tools.

Topics Covered

  • Stanford-specific and external resources for data management
  • Command line data transfer tools:
    • rsync (needed for FarmShare/Sherlock/OAK)
    • rclone (used for cloud services)

This workshop explores how to integrate Large Language Models into scientific research. You will begin by learning the basics of how LLMs work. You will then gain hands-on experience generating a retrieval-augmented (RAG) model that can query and discuss large volumes of scientific knowledge without hallucination.

Topics Covered

  • History, development, and current state of large language models (LLMs)
  • Building a retrieval-augmented (RAG) model that allows for management and querying of scientific knowledge

This workshop will teach attendees how to deploy portable, reproducible computational environments that support interactive data analysis while using HPC resources. As an example, you will be able to work in Jupyter while using Sherlock and always have the right python environment for your project.

Topics Covered

  • Promoting reproducibility through using containerized analysis
  • Interactive, containerized, brower-based analysis using HPC resources:
    • FarmShare/Sherlock OnDemand
    • Hosting python kernels on FarmShare/Sherlock

This workshop focuses on creating and sharing custom computational environments. You will learn to build containers from scratch and distribute them to your lab or the public, ensuring your workflows run exactly the same way on any machine.

Topics Covered

  • Modifying existing containers
  • Building custom containers from scratch
  • Making custom containers available to your lab or to the public

This workshop will introduce attendees to the concept of embarrassingly parallel analysis problems and how to approach them efficiently with parallelization at scale. This is especially useful for anyone working with time series data.

Topics Covered

  • Introduction to embarrassingly parallel problems
  • Methods of parallelization at scale:
    • GNU Parallel (powerful, shell-based tool that allows for easy parallization of tasks on a local machine)
    • Pub/Sub (cloud-based service that allows for scalable, aynchronous analysis pipelines)

This workshop brings everything together by teaching you how to build automated, reproducible scientific data pipelines. You will learn to use GitLab CI to automate your analysis stages, ensuring that your research results are always reproducible and verified by continuous integration.

Topics Covered

  • Fundamentals of continuous integration and pipelines
  • GitLab CI