Using Jupyter Hub
Jupyter provides an interactive computing environment for different languages. The most popular use case is Python; Project Jupyter is a spin-off of the interactive Python (iPython) project.
With Jupyter Hub, you can run Python and R notebooks, or open a terminal directly on your browser.
- You can log in to Theta or Cooley instances of Jupyter Hub at https://jupyter.alcf.anl.gov using your Theta/Cooley credentials.
- Theta Jupyter Hub instances runs on an external server (jupyter02.mcp.alcf.anl.gov), but not on Theta login, mom, or compute nodes.
- Theta Jupyer Hub has access to your home folder (
/home/$USER
) and projects folder (/projects
), but does not have access to/opt/cray
,/opt/intel
, etc.—that is, you cannot use any Theta modules or any Cray libraries. - Cooley Jupyter Hub instances have access to the user's home folder (/home/$USER), the Mira projects folder (/projects), and the Theta project folder (/lus/theta-fs0/projects).
- You can submit jobs to run on Theta/Cooley with
!qsub myjob.sh
. - You can customize your environment and add new kernels. To do so, you could run shell commands on a notebook cell by prepending them with
!
(i.e.,!pwd
). However, it would be much easier to use the Jupyter terminal. You can open a terminal on the Jupyter Hub home page under Files/new menu. Using the terminal, you can create a new conda environment and a new Jupyter kernel with the commands below.
For more information on how to manage conda environments, refer to this page.
From terminal:
conda create -n jhub_myenv source activate jhub_myenv conda install jupyter nb_conda ipykernel conda install any_module_you_need source deactivate
From notebook:
!conda create -y -n jhub_myenv !source activate jhub_myenv; conda install -y jupyter nb_conda ipykernel !source activate jhub_myenv; conda install -y any_module_you_need
This will complete the installation of a new environment. To create a Jupyter kernel, additional work is necessary:
From terminal:
source activate jhub_myenv python -m ipykernel install --user --name jhub_myenv source deactivate
From notebook:
!source activate;python -m ipykernel install --user --name jhub_myenv
After completing these steps, you will see jhub_myenv
kernel when you click new
on the Jupyter Hub home page or when you use Kernel menu on a Jupyter notebook.
- Use a name identifier (such as
jhub_
prefix used above) on your Jupyter Hub conda environment to make sure you don’t mix them with conda environments you create on Cooley/Theta nodes.
Accessing Project Folders:
Jupyterhub limits to a user home directory the user's ability to browse the file system. To access a given project directory, the user must create a symbolic link in their home directory to the project directory:
If a user is connected to the Cooley instances and wants to access Mira project ABC and Theta project EFG, the user should execute the following commands to create the links:
From terminal:
ln -s /project/ABC ABC_project ln -s /lus/theta-fs0/projects/EFG EFG_project
From notebook:
!ln -s /project/ABC ABC_project !ln -s /lus/theta-fs0/projects/EFG EFG_project
Hardware
- CPU : Intel(R) Xeon(R) CPU E5-2683
- RAM : 512GB
- GPU: None