{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Jupyter Notebooks " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Acknowledgements" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The material in this tutorial is specific to PYNQ. Wherever possible, however, it re-uses generic documentation describing Jupyter notebooks. In particular, we have re-used content from the following example notebooks:\n", "\n", "1. What is the Jupyter Notebook?\n", "1. Notebook Basics\n", "1. Running Code\n", "1. Markdown Cells\n", "\n", "The original notebooks and further example notebooks are available at [Jupyter documentation](http://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/examples_index.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introduction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you are reading this documentation from the webpage, you should note that the webpage is a static html version of the notebook from which it was generated. If the PYNQ platform is available, you can open this notebook from the getting_started folder in the PYNQ Jupyter landing page. \n", "\n", "The Jupyter Notebook is an **interactive computing environment** that enables users to author notebook documents that include:\n", "\n", "* Live code\n", "* Interactive widgets\n", "* Plots\n", "* Narrative text\n", "* Equations\n", "* Images\n", "* Video\n", "\n", "These documents provide a **complete and self-contained record of a computation** that can be converted to various formats and shared with others electronically, using version control systems (like git/[GitHub](http://github.com)) or [nbviewer.jupyter.org](http://nbviewer.jupyter.org)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Components" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Jupyter Notebook combines three components:\n", "\n", "* **The notebook web application**: An interactive web application for writing and running code interactively and authoring notebook documents.\n", "* **Kernels**: Separate processes started by the notebook web application that runs users' code in a given language and returns output back to the notebook web application. The kernel also handles things like computations for interactive widgets, tab completion and introspection. \n", "* **Notebook documents**: Self-contained documents that contain a representation of all content in the notebook web application, including inputs and outputs of the computations, narrative text, equations, images, and rich media representations of objects. Each notebook document has its own kernel." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Notebook web application" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The notebook web application enables users to:\n", "\n", "* **Edit code in the browser**, with automatic syntax highlighting, indentation, and tab completion/introspection.\n", "* **Run code from the browser**, with the results of computations attached to the code which generated them.\n", "* See the results of computations with **rich media representations**, such as HTML, LaTeX, PNG, SVG, PDF, etc.\n", "* Create and use **interactive JavaScript widgets**, which bind interactive user interface controls and visualizations to reactive kernel side computations.\n", "* Author **narrative text** using the [Markdown](https://daringfireball.net/projects/markdown/) markup language.\n", "* Build **hierarchical documents** that are organized into sections with different levels of headings.\n", "* Include mathematical equations using **LaTeX syntax in Markdown**, which are rendered in-browser by [MathJax](http://www.mathjax.org/)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Kernels" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Notebook supports a range of different programming languages. For each notebook that a user opens, the web application starts a kernel that runs the code for that notebook. Each kernel is capable of running code in a single programming language. There are kernels available in the following languages:\n", "\n", "* Python https://github.com/ipython/ipython\n", "* Julia https://github.com/JuliaLang/IJulia.jl\n", "* R https://github.com/takluyver/IRkernel\n", "* Ruby https://github.com/minrk/iruby\n", "* Haskell https://github.com/gibiansky/IHaskell\n", "* Scala https://github.com/Bridgewater/scala-notebook\n", "* node.js https://gist.github.com/Carreau/4279371\n", "* Go https://github.com/takluyver/igo\n", "\n", "PYNQ is written in Python, which is the default kernel for Jupyter Notebook, and the only kernel installed for Jupyter Notebook in the PYNQ distribution. \n", "\n", "Kernels communicate with the notebook web application and web browser using a JSON over ZeroMQ/WebSockets message protocol that is described [here](http://ipython.org/ipython-doc/dev/development/messaging.html). Most users don't need to know about these details, but its important to understand that kernels run on Zynq, while the web browser serves up an interface to that kernel." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Notebook Documents" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notebook documents contain the **inputs and outputs** of an interactive session as well as **narrative text** that accompanies the code but is not meant for execution. **Rich output** generated by running code, including HTML, images, video, and plots, is embedded in the notebook, which makes it a complete and self-contained record of a computation. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When you run the notebook web application on your computer, notebook documents are just **files** on your local filesystem with a **.ipynb** extension. This allows you to use familiar workflows for organizing your notebooks into folders and sharing them with others." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notebooks consist of a **linear sequence of cells**. There are four basic cell types:\n", "\n", "* **Code cells:** Input and output of live code that is run in the kernel\n", "* **Markdown cells:** Narrative text with embedded LaTeX equations\n", "* **Heading cells:** Deprecated. Headings are supported in Markdown cells\n", "* **Raw cells:** Unformatted text that is included, without modification, when notebooks are converted to different formats using nbconvert\n", "\n", "Internally, notebook documents are [JSON](http://en.wikipedia.org/wiki/JSON) data with binary values [base64](http://en.wikipedia.org/wiki/Base64) encoded. This allows them to be **read and manipulated programmatically** by any programming language. Because JSON is a text format, notebook documents are version control friendly.\n", "\n", "**Notebooks can be exported** to different static formats including HTML, reStructeredText, LaTeX, PDF, and slide shows ([reveal.js](http://lab.hakim.se/reveal-js/#)) using Jupyter's `nbconvert` utility. Some of documentation for Pynq, including this page, was written in a Notebook and converted to html for hosting on the project's documentation website. \n", "\n", "Furthermore, any notebook document available from a **public URL or on GitHub can be shared** via [nbviewer](http://nbviewer.ipython.org). This service loads the notebook document from the URL and renders it as a static web page. The resulting web page may thus be shared with others **without their needing to install the Jupyter Notebook**.\n", "\n", "GitHub also renders notebooks, so any Notebook added to GitHub can be viewed as intended. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Notebook Basics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The Notebook dashboard" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Notebook server runs on the ARM® processor of the board. You can open the notebook dashboard by navigating to [pynq:9090](http://pynq:9090) when your board is connected to the network. \n", "The dashboard serves as a home page for notebooks. Its main purpose is to display the notebooks and files in the current directory. For example, here is a screenshot of the dashboard page for an example directory:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Dashboard](../images/dashboard_files_tab.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The top of the notebook list displays clickable breadcrumbs of the current directory. By clicking on these breadcrumbs or on sub-directories in the notebook list, you can navigate your filesystem.\n", "\n", "To create a new notebook, click on the \"New\" button at the top of the list and select a kernel from the dropdown (as seen below)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Dashboard with new file](../images/dashboard_files_tab_new.JPG)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notebooks and files can be uploaded to the current directory by dragging a notebook file onto the notebook list or by the \"click here\" text above the list.\n", "\n", "The notebook list shows green \"Running\" text and a green notebook icon next to running notebooks (as seen below). Notebooks remain running until you explicitly shut them down; closing the notebook's page is not sufficient." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Dashboard run](../images/dashboard_files_tab_run.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To shutdown, delete, duplicate, or rename a notebook check the checkbox next to it and an array of controls will appear at the top of the notebook list (as seen below). You can also use the same operations on directories and files when applicable." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Dashboard controls](../images/dashboard_files_tab_btns.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To see all of your running notebooks along with their directories, click on the \"Running\" tab:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Dashboard running](../images/dashboard_running_tab.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This view provides a convenient way to track notebooks that you start as you navigate the file system in a long running notebook server." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Overview of the Notebook UI" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you create a new notebook or open an existing one, you will be taken to the notebook user interface (UI). This UI allows you to run code and author notebook documents interactively. The notebook UI has the following main areas:\n", "\n", "* Menu\n", "* Toolbar\n", "* Notebook area and cells\n", "\n", "The notebook has an interactive tour of these elements that can be started in the \"Help:User Interface Tour\" menu item." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Modal editor" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Jupyter Notebook has a modal user interface which means that the keyboard does different things depending on which mode the Notebook is in. There are two modes: edit mode and command mode." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Edit mode" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Edit mode is indicated by a green cell border and a prompt showing in the editor area:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Edit Mode](../images/edit_mode.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When a cell is in edit mode, you can type into the cell, like a normal text editor." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "Enter edit mode by pressing `Enter` or using the mouse to click on a cell's editor area.\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Command mode" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Command mode is indicated by a grey cell border with a blue left margin:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Command mode](../images/command_mode.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When you are in command mode, you are able to edit the notebook as a whole, but not type into individual cells. Most importantly, in command mode, the keyboard is mapped to a set of shortcuts that let you perform notebook and cell actions efficiently. For example, if you are in command mode and you press `c`, you will copy the current cell - no modifier is needed.\n", "\n", "
\n", "Don't try to type into a cell in command mode; unexpected things will happen!\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "Enter command mode by pressing `Esc` or using the mouse to click *outside* a cell's editor area.\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Mouse navigation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "All navigation and actions in the Notebook are available using the mouse through the menubar and toolbar, both of which are above the main Notebook area:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Menu toolbar](../images/menubar_toolbar.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Cells can be selected by clicking on them with the mouse. The currently selected cell gets a grey or green border depending on whether the notebook is in edit or command mode. If you click inside a cell's editor area, you will enter edit mode. If you click on the prompt or output area of a cell you will enter command mode.\n", "\n", "If you are running this notebook in a live session on the board, try selecting different cells and going between edit and command mode. Try typing into a cell." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you want to run the code in a cell, you would select it and click the `play` button in the toolbar, the \"Cell:Run\" menu item, or type Ctrl + Enter. Similarly, to copy a cell you would select it and click the `copy` button in the toolbar or the \"Edit:Copy\" menu item. Ctrl + C, V are also supported.\n", "\n", "Markdown and heading cells have one other state that can be modified with the mouse. These cells can either be rendered or unrendered. When they are rendered, you will see a nice formatted representation of the cell's contents. When they are unrendered, you will see the raw text source of the cell. To render the selected cell with the mouse, and execute it, click the `play` button in the toolbar or the \"Cell:Run\" menu item, or type Ctrl + Enter. To unrender the selected cell, double click on the cell." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Keyboard Navigation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are two different sets of keyboard shortcuts: one set that is active in edit mode and another in command mode.\n", "\n", "The most important keyboard shortcuts are `Enter`, which enters edit mode, and `Esc`, which enters command mode.\n", "\n", "In edit mode, most of the keyboard is dedicated to typing into the cell's editor. Thus, in edit mode there are relatively few shortcuts. In command mode, the entire keyboard is available for shortcuts, so there are many more. The `Help`->`Keyboard Shortcuts` dialog lists the available shortcuts." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Some of the most useful shortcuts are:\n", "\n", "1. Basic navigation: `enter`, `shift-enter`, `up/k`, `down/j`\n", "2. Saving the notebook: `s`\n", "2. Change Cell types: `y`, `m`, `1-6`, `t`\n", "3. Cell creation: `a`, `b`\n", "4. Cell editing: `x`, `c`, `v`, `d`, `z`\n", "5. Kernel operations: `i`, `0` (press twice)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Running Code" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First and foremost, the Jupyter Notebook is an interactive environment for writing and running code. The notebook is capable of running code in a wide range of languages. However, each notebook is associated with a single kernel. Pynq, and this notebook is associated with the IPython kernel, which runs Python code." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Code cells allow you to enter and run code" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Run a code cell using `Shift-Enter` or pressing the `play` button in the toolbar above. The button displays *run cell, select below* when you hover over it." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "a = 10" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "print(a)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are two other keyboard shortcuts for running code:\n", "\n", "* `Alt-Enter` runs the current cell and inserts a new one below.\n", "* `Ctrl-Enter` run the current cell and enters command mode." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Managing the Kernel" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Code is run in a separate process called the Kernel. The Kernel can be interrupted or restarted. Try running the following cell and then hit the `stop` button in the toolbar above. The button displays *interrupt kernel* when you hover over it." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import time\n", "time.sleep(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Cell menu" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The \"Cell\" menu has a number of menu items for running code in different ways. These includes:\n", "\n", "* Run and Select Below\n", "* Run and Insert Below\n", "* Run All\n", "* Run All Above\n", "* Run All Below" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Restarting the kernels" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The kernel maintains the state of a notebook's computations. You can reset this state by restarting the kernel. This is done from the menu bar, or by clicking on the corresponding button in the toolbar." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### sys.stdout " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The stdout and stderr streams are displayed as text in the output area." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "print(\"Hello from Pynq!\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Output is asynchronous" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "All output is displayed asynchronously as it is generated in the Kernel. If you execute the next cell, you will see the output one piece at a time, not all at the end." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import time, sys\n", "for i in range(8):\n", " print(i)\n", " time.sleep(0.5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Large outputs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To better handle large outputs, the output area can be collapsed. Run the following cell and then single- or double- click on the active area to the left of the output:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "scrolled": true }, "outputs": [], "source": [ "for i in range(50):\n", " print(i)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Markdown" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Text can be added to Jupyter Notebooks using Markdown cells. Markdown is a popular markup language that is a superset of HTML. Its specification can be found here:\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Markdown basics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can make text *italic* or **bold**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can build nested itemized or enumerated lists:\n", "\n", "* One\n", " - Sublist\n", " - This\n", " - Sublist\n", " - That\n", " - The other thing\n", "* Two\n", " - Sublist\n", "* Three\n", " - Sublist\n", "\n", "Now another list:\n", "\n", "1. Here we go\n", " 1. Sublist\n", " 2. Sublist\n", "2. There we go\n", "3. Now this" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can add horizontal rules:\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here is a blockquote:\n", "\n", "> Beautiful is better than ugly.\n", "> Explicit is better than implicit.\n", "> Simple is better than complex.\n", "> Complex is better than complicated.\n", "> Flat is better than nested.\n", "> Sparse is better than dense.\n", "> Readability counts.\n", "> Special cases aren't special enough to break the rules.\n", "> Although practicality beats purity.\n", "> Errors should never pass silently.\n", "> Unless explicitly silenced.\n", "> In the face of ambiguity, refuse the temptation to guess.\n", "> There should be one-- and preferably only one --obvious way to do it.\n", "> Although that way may not be obvious at first unless you're Dutch.\n", "> Now is better than never.\n", "> Although never is often better than *right* now.\n", "> If the implementation is hard to explain, it's a bad idea.\n", "> If the implementation is easy to explain, it may be a good idea.\n", "> Namespaces are one honking great idea -- let's do more of those!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And shorthand for links:\n", "\n", "[Jupyter's website](http://jupyter.org)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Headings" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can add headings by starting a line with one (or multiple) `#` followed by a space, as in the following example:\n", "\n", "```\n", "# Heading 1\n", "# Heading 2\n", "## Heading 2.1\n", "## Heading 2.2\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Embedded code" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can embed code meant for illustration instead of execution in Python:\n", "\n", " def f(x):\n", " \"\"\"a docstring\"\"\"\n", " return x**2\n", "\n", "or other languages:\n", "\n", " if (i=0; i\n", " ```python\n", " print \"Hello World\"\n", " ```\n", " \n", "\n", " 
\n",
    "    ```javascript\n",
    "    console.log(\"Hello World\")\n",
    "    ```\n",
    "
\n", "\n", "Gives:\n", "\n", "```python\n", "print \"Hello World\"\n", "```\n", "\n", "```javascript\n", "console.log(\"Hello World\")\n", "```\n", "\n", "And a table like this: \n", "\n", " 
\n",
    "    ```\n",
    "\n",
    "    | This | is   |\n",
    "    |------|------|\n",
    "    |   a  | table| \n",
    "\n",
    "    ```\n",
    "
\n", "\n", "A nice HTML Table:\n", "\n", "| This | is |\n", "|------|------|\n", "| a | table| \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### General HTML" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because Markdown is a superset of HTML you can even add things like HTML tables:\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Header 1Header 2
row 1, cell 1row 1, cell 2
row 2, cell 1row 2, cell 2
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Local files" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you have local files in your Notebook directory, you can refer to these files in Markdown cells directly:\n", "\n", " [subdirectory/]\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Security of local files" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the Jupyter notebook server also acts as a generic file server\n", "for files inside the same tree as your notebooks. Access is not granted outside the\n", "notebook folder so you have strict control over what files are visible, but for this\n", "reason it is highly recommended that you do not run the notebook server with a notebook\n", "directory at a high level in your filesystem (e.g. your home directory).\n", "\n", "When you run the notebook in a password-protected manner, local file access is restricted\n", "to authenticated users unless read-only views are active. For more information, see Jupyter's\n", "documentation on [running a notebook server](https://jupyter-server.readthedocs.io/en/latest/operators/public-server.html).\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.0" } }, "nbformat": 4, "nbformat_minor": 1 }