Introduction to Jupyter Notebooks

Table of Contents

• Background
• Short Cuts
• Markdown
• LaTeX
• Table of Contents in the Notebook
• Jupyter Lab
• Notebooks in Education
• Final Words

Background

Jupyter notebooks create a great workflow for developing and sharing code. While we might often turn to editors (e.g., Emacs, Vim, Atom, nano, etc.) or integrated development environments (IDEs; e.g., Spyder, Canopy, Eclipse, etc.), a broad range of more sophisticated environments have been developed across various platforms. For example, you might be familiar with the look and feel of Mathematica or Matlab; these differ from basic editors in that they provide an immersive graphical environment for rapid prototyping of code. Where do Jupyter notebooks fit into this picture?

Jupyter Notebooks (JNBs) were designed to gather in one tool the best features of all of these environments. And, they can be used with a range of languages (about 40, as of the time of this writing); the original version was developed for Python, which was called an “iPython notebook”. The name “Jupyter” refers to Julia (Ju), Python (pyt), and R (r), but these three languages are just the tip of the iceberg. (Most of my machines are set up to use Python 2, Python 3 and Julia;  I simply select which I need for that new notebook.) Here are just some of the things you can do with JNBs:

• Access the operating system while you are working, such as changing directories and looking for files.
• Embed videos, links to other websites, pictures, etc.
• Time portions of code to find speed bottlenecks.
• Very easily and beautifully comment using text cells. This is extremely powerful, using the full features of HTML and CSS, so your JNB can incorporate all of the advances features you see at websites. Commenting code becomes enjoyable!
• But, using the rapid features of markdown, you can also create quite nice documentation without the need for full HTML/CSS. Don’t be intimidated! (And, see the introduction to Markdown below.)
• Include mathematical formulae as part of the code documentation, using the full capabilities of LaTeX.
• Collaborate and communicate much more efficiently. Even if your collaborators don’t use JNBs (perhaps you should choose better collaborators!), you can send them an HTML version of your JNB that displays, but doesn’t need to run, in their browser.
• Use in educational settings, with lecture/instruction directly included with the code. Many of the classes we teach here in the CMSE department are JNB based (see below).

Okay, surely you are convinced that JNBs will be part of your future workflow. What next? Notebooks run in your browser but you will need to install the tools on your computer. In fact, your computer may already have the capability if you have a modern and complete installation of one of the languages it supports. For example, if you use the Continuum Analytics or Enthought Python distributions, you already have what you need. The simplest way to start a JNB, which will appear in your browser, is at the command line with ‘jupyter notebook’, but you can often get there through the IDEs (e.g., Anaconda Navigator, Canopy, etc.) that come with these distributions. If you don’t have what you need already, read this page, which has information for a wide range of settings. If you plan to use Julia, this is a great page.

The basic idea of a JNB is cells of different types. The main type of cell is a code cell, but text-based cells make your code very readable. Here is what an example JNB looks like:

Let’s take a look at what is here. The first cell contains some formatted text, which was done in markdown (see below). This is really useful to refresh the reader about the notebook, and that might be you! The next two blocks contain some Python code, and they are both commented using markdown. Then you can see an equation, done in a markdown cell, but using LaTeX to make the equation appear as you would expect in a book. Finally, a little more code is given that produces the plot. This is a random example I threw together, but hopefully you see the idea: where you write your code can also include a lot of other helpful things.

Now, before you create a notebook like the one above, you will likely be taken to a control page before the actual JNB, which might look like this:

From this launch page you can do many things, but notice on the upper right you can start a new JNB with the language of your choice (assuming you have things set up correctly!) or simply click on a previous file to start where you left off. Note that I have Python 3 and Julia 0.4.6 – your choices are very likely to be different. Select from your list and it will create an empty notebook like the one above.

I also wrote a tutorial on notebooks in the form of a notebook itself. You can download that from here.

Short Cuts

• Your workflow will be much faster if you don’t use the menus at the top of the JNB. Use the keyboard shortcuts! Here’s how:
  • Note that a cell has two levels of activation: the outer box around the cell (Command Mode) and inside the cell itself (Edit Mode). The border will change color depending on which is selected. It can be frustrating when you start typing when you think you are in Edit Mode, and you have accidentally selected Command Mode.
  • But, use this feature to your advantage! When you need to make new cells, select Command Mode so that the border is blue; now, you can give that cell commands which will not appear in the cell itself; this is why there are two modes. I get to Command Mode by clicking outside the cell on the left side where the cell numbers or by pressing Esc.
  • Now, once in Command Mode, you can type a wide range of single letters that do things very quickly:
    • [Shift+Enter]: run that cell (and select the cell below it; or, use [Control+Enter] if you want to stay in that cell); this works even if you are in Edit Mode
    • x: delete that cell
    • a: create a new cell above
    • b: create a new cell below
    • m: make that cell a markdown cell
    • l: add line numbers (useful when working with others)
    • y: make that cell a code cell (which is the default for a new cell)
    •  [Enter] : enter Edit Mode (turning the cell green and putting the cursor inside)
    • ↑ ↓: use the up and down arrows to move through the cells
    •  [Number] : type a number (i.e., 1, 2, 3,…) to add a heading with that size (then use Enter to get into Edit Mode)
  • As an example, suppose you want to add a comment above a code cell you are working in. Type: to enter Command Mode, a to add above, m for markdown, and to get into Edit Mode – starting typing your comment! No mouse, no menus!
• To keep your Matplotlib plots inside the notebook (rather than popping out into a separate GUI window; see example above), use either of these:

  • %matplotlib inline

  • %matplotlib notebook

• Many people run each cell as they are debugging that cell, a great way to ensure that each piece of code is working before moving on. But, this means that you might be executing cells in random orders. Be sure that you run the cells above the cell you are working on if there are any dependencies, such as preparing data in some way. There is discussion of adding a “run all” feature, but it isn’t there as of this writing.
• Although these shortcuts make your workflow much faster, it is still a good idea to walk through all of the menus at the top just so you know what is there; the list of shortcuts above is not exhaustive of all the things you can do.

Markdown

As I mentioned, one of the best features of JNBs is that you will actually want to comment your code! Unless you are an expert writing in HTML, I would suggest starting with markdown. The basic steps are these: make a new cell, make that cell a markdown cell (as described above or with the menu at the top), enter Edit Mode, start typing, run that cell (press Shift+Enter). Simple. Now, you need to know a few basic markdown tricks:

• #, ##, ###, ####: put these at the beginning of the line to make headings of different sizes; intuitively, more #s means smaller font
• bold: use two astericks **my bold text** or double underscore __my bold text__
• italics: use single asterick *my italics text** or underscore _my italics text_
• strikethrough: use ~~I think we don’t need this anymore~~
• make a bulleted list (like this one!) by placing each item on its own line with * at the beginning of the line; you can also number them and it knows what you are doing in both cases
• For major sections, such as the title section or a function, I like to have horizontal lines top and bottom; an easy way to do this uses three underscores, as in ___, which will render as a line across the full width.
• To quickly edit a markdown cell that has been executed, simply double click on the cell to make the markdown reappear.
• inline links: place text that will be the link in [] followed by the link itself in (), as in

  [Find Murillo Group!](https://www.murillogroupmsu.com)

LaTeX

Most of us will use the JNB in the context of some scientific setting: documentation of the code might involve equations. The standard typesetting tool is LaTeX, and it is worth learning if you don’t know it already. Unfortunately, it has a bit of a learning curve and is like learning yet another language. The flip side is that you get very high quality equations in your notebooks, you can transfer your knowledge to other settings (such as LaTeXiT), you will end up using only a few commands in most of your work and you can search and copy vast numbers of examples from around the internet. If you are new to LaTeX, here is a handy guide.

If you would like to use an entire cell for LaTeX, start the cell with:
%%latex. However, I suspect this is not a typical part of your workflow, and you will want to insert equations with the text. Inside of a sentence, simple use dollar signs, as in $E=mc^2$. Or, between sentences, on a blank line in the cell place $$ on either wide of the equation, as in $$F = ma$$; this will display the equation on its own line and centered. See the example above for an example of this latter type.

I like the Notebook+LaTeX environment so much that I have used it to quickly write some notes with no code to share with collaborators. Once you get used to this workflow, you might not use other LaTeX-centric GUIs (e.g., TeXShop) anymore.

Adding a Table of Contents

It’s definitely worth learning a little HTML coding if you are interested in getting serious about JNBs. While Markdown is really great, it currently is somewhat limited and you will likely want to add a few more tricks to your toolbox; HTML takes you the rest of the way. While there is a lot to learn, here is one example.

One of the most useful HTML capabilities to use in Notebooks is internal linking, such as adding a table of contents; this is especially useful if you write very long notebooks. There are two ways to do this: (1) using standard HTML, or (2) using the markdown capabilities within the Notebook. In this webpage, for example, I have used the HTML version to allow you to jump to this section from the top of the page. Either way, this works by adding two lines of codes: one where you want to jump from and one where you want to jump to; these are connected by some HTML code (somewhat like LaTeX if you have used that.)

Let’s use as an example the link at the top of the page. Note that the text that forms the link is between two blocks of the form ; that is how it knows which words to use, and not others, for the link. The code “#HTML” tells HTML to look for “HTML” somewhere else in the page to jump to. Here is what that first bit of coding might look like:

<href="#HTML">Quick HTML Tips for Notebooks 

This can be a bit simpler using markdown, which has the form

[Quick HTML Tips for Notebooks](#HTML) ,

which is perhaps a bit easier to remember in the beginning.
Next, we include this coding at the point we want to jump to, which includes our chosen code “HTML”.

If you would like to learn about HTML, one of the best sites I have found is w3schools, which is very complete tutorial. And, it is not just HMTL you can use, but also the full power of cascading style sheets (CSS). For example, you can change the theme of your Notebook, making it light on dark.

If you would like to use an entire cell for HTML, start the cell with:
%%HTML.

A very nice idea for large JNBs is to add links at key places throughout that allow you to jump back up to the table of contents. For example, because your code will be organized into functions, classes, etc., you might add a “Return to TOC” link after each one so that the reader can find their next destination in your code more easily without scrolling back to the top.

Jupyter Lab

Interactive Development Environments (IDEs) are a popular tool for developing code. Most IDEs present a frame that is subdivided into smaller frames that contain easy access to the file/directory system, tabs containing codes, perhaps a command interface to the operating system and maybe a pane where output (e.g., plots) appear. This work is environment is very efficient and quite different from the Jupyter notebook. Which should you pick?!

It turns out that you no longer need to decide: the new Jupyter Lab provides both. Jupyter Lab provides a browser-based IDE which has tabs that contain Jupyter notebooks; you now get the best of both worlds. Jupyter Lab is currently only in alpha, so you might consider waiting until it is a bit more developed. I have been using it for a few months and find it to be a bit clunky; and, upgrades often improve some things and break other things. Nevertheless, I am working to make it part of my daily workflow even if I need to switch back to the basic notebook when real work needs to get done (such as preparing an assignment). The problems tend to be minor: text color in a markdown cell, for example.

How can you get and use Jupyter Lab? It’s easy: everything you need is at this website, where you see that you simply need to do an install (e.g., using pip) and call “jupyter lab” from a command line. Here is a short video to show you how to move around in the IDE:

For me the two most important features are the access to the file system with the ability to search and click on files and having multiple tabs open simultaneously that I can quickly flip and copy between. See what you think!

Education

JNBs are fantastic in the context of education and we use them extensively in our department. For the student:

• Code and its full documentation are combined into one document, including results in the form of visualizations.
• Documentation is high quality: equations, bullets, bold/italic, headers, line separators, links and images/videos.
• Cells encourage and simplify debugging techniques.
• Language specific text highlighting.

Similarly, JNBs are of great use to the instructor:

• Assignments can be written as a JNB using Markdown.
• Background information can be included in most forms: embedded video, images, links to Wikipedia, etc.
• Google forms can be embedded to gather common information from the students in an organized manner (e.g., feedback on the assignment).
• The assignment that is sent out is completed and returned as one notebook.
• Code can be scaffolded: the notebook can be prepared with partial code, explained in Markdown, and left to be completed by the student.

Contact us if you would like to learn more about using JNBs in your classroom!

Final Words

You should now have everything you need to get started with JNBs! What is here may be all you ever need. But, it is also just the start of a potentially long and rewarding journey. If you want to go further, let me point you to just a few exciting advanced features of JNBs:

• Interactive plots using Bokeh.
• Animations.
• Widgets, and more widgets.
• Share!
• Include or invent a theme!
  1. pip install –upgrade pip
  2. pip install jupyter-themer
  3. jupyter-themer -c base16-dark -b dark
  4. reload all of your JNBs!
  5. use just jupyter-themer with no arguments to reset to default

And, keep in mind that JNBs are under constant development. Keep an eye out for new features. For example, think of creative ways you can communicate things by including these new tools, or even just a figure: