Installing the Python3 and Go Kernels in Jupyter

Earlier we mentioned that Jupyter Notebook is an interactive notebook that supports running more than 40 programming languages and is very well suited to teaching. Recently, while learning the Go language, I thought of Jupyter. This article mainly describes how to install the Python3 and Go kernels inside Jupyter.

1. Installing Jupyter

On CentOS 7, Python 2.7 is installed by default, so there is a ready-made Python environment to work with.

Install Jupyter

1
pip install jupyter

Generate a password

In IPython, use the passwd function to generate a password sha1 value.

ipython
In [1]: from IPython.lib import passwd

In [2]: passwd()
Enter password:
Verify password:
Out[2]: 'sha1:************************************************'

Generate the config file

Run the command jupyter notebook --generate-config to generate the ~/.jupyter/jupyter_notebook_config.py config file.

Edit the jupyter_notebook_config file and add the following content:

1
2
3
4
5
c.NotebookApp.allow_root = True
c.NotebookApp.open_browser = False
c.NotebookApp.port = 9999
c.NotebookApp.password = u'sha1:************************************************''
c.ContentsManager.root_dir = '/root/jupyter/data'

Here, passwd is the sha1 value of the password generated just now, and root_dir is the root directory where Jupyter Notebook data is stored.

Start Jupyter

At this point the installation and configuration of Jupyter are essentially complete, and you can start Jupyter.

1
jupyter notebook

Error when started as a non-root user

If you start it without root privileges, an error is reported: OSError: [Errno 13] Permission denied: ‘/run/user/0/jupyter’. This is caused by a lack of permissions. You only need to use virtualenv in a directory you have permission for to create a Python environment at /home/jupyter/py2.

Run the command to modify the XDG_RUNTIME_DIR value:

1
export XDG_RUNTIME_DIR="/home/jupyter/py2/jupyter"

Check the runtime path:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
jupyter --path
config:
    /root/.jupyter
    /usr/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /root/.local/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /home/jupyter/py2/jupyter

2. Managing Jupyter with supervisor

When running Jupyter on a server, in order to bring it back up automatically when the process dumps, you need to hand Jupyter over to supervisor.

Install supervisor

1
pip install supervisor

Create the config file directory

1
mkdir -p /etc/supervisor/conf.d

Create the supervisor config file

1
echo_supervisord_conf > /etc/supervisor/supervisord.conf

Edit the supervisor config file

Edit the /etc/supervisor/supervisord.conf file and add the following content:

1
2
[include]
files = /etc/supervisor/conf.d/*.ini

Every config file under the /etc/supervisor/conf.d/ directory will then be imported. This is for ease of management: you can write a separate config file for each process, or for a few related processes.

Add the Jupyter config jupyter.ini

Add the file /etc/supervisor/conf.d/jupyter.ini with the following content:

1
2
3
4
5
6
7
8
9
[program:jupyter]
command=jupyter notebook --allow-root
user=root
autostart=true
;autorestart=true
;startsecs=30
;startretries=5
stdout_logfile=/var/log/jupyter/jupyter-access.log
stderr_logfile=/var/log/jupyter/jupyter-error.log

Create the log directory

1
mkdir /var/log/jupyter

Start supervisor

You can simply run this to start supervisor:

1
supervisord

Or you can start it with a specified config file

1
/usr/bin/supervisord -c /etc/supervisord.conf

Check status with supervisorctl

supervisorctl is a program used to interact with supervisord. Through supervisorctl, we can manage the processes being supervised. Here are some basic commands:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# 停止某一个进程，program_name 为 [program:x] 里的 x
supervisorctl stop program_name
# 启动某个进程
supervisorctl start program_name
# 重启某个进程
supervisorctl restart program_name
# 结束所有属于名为 groupworker 这个分组的进程 (start，restart 同理)
supervisorctl stop groupworker:
# 结束 groupworker:name1 这个进程 (start，restart 同理)
supervisorctl stop groupworker:name1
# 停止全部进程，注：start、restart、stop 都不会载入最新的配置文件
supervisorctl stop all
# 载入最新的配置文件，停止原有进程并按新的配置启动、管理所有进程
supervisorctl reload
# 根据最新的配置文件，启动新配置或有改动的进程，配置没有改动的进程不会受影响而重启
supervisorctl update
# 查看进程状态
supervisorctl status

3. Configuring the Go Kernel in Jupyter

gophernotes is the Go kernel for Jupyter. The figure below shows an example of running Go in Jupyter after the kernel is configured:

Install the dependencies

gophernotes has four dependencies:

Go 1.9+
Jupyter Notebook or nteract
ZeroMQ 4.X.X
pkg-config

On CentOS 7 you only need to install ZeroMQ and Go to satisfy the dependencies.

1
yum install zeromq go

Install gopherdata

1
2
3
go get -u github.com/gopherdata/gophernotes
mkdir -p ~/.local/share/jupyter/kernels/gophernotes
cp $GOPATH/src/github.com/gopherdata/gophernotes/kernel/* ~/.local/share/jupyter/kernels/gophernotes

gophernotes not found because it is not on the PATH

Error message: OSError: [Errno 2] No such file or directory

1
ln -s $HOME/go/bin/gophernotes /go/bin/gophernotes

You can create the symlink as suggested on GitHub, or simply copy gophernotes into the /usr/bin/ directory.

4. Configuring the Python3 Kernel in Jupyter

After configuring Go, I went ahead and configured Python3 as well. The figure below shows an example of running Python3 in Jupyter after the kernel is configured:

Check the available Python3 versions

1
yum  search python3

Install Python36

1
yum install python36

Install pip3

1
2
wget https://bootstrap.pypa.io/get-pip.py
python36 get-pip.py

Install the kernel

python36 -m pip install ipykernel
python36 -m ipykernel install

Error when widgetsnbextension is not installed and enabled

The frontend reports a 404 error, unable to find /jupyter/nbextensions/widgets/notebook/js/extension.js

Just install and enable widgetsnbextension.

1
2
jupyter nbextension install --py widgetsnbextension
jupyter nbextension enable widgetsnbextension --py

5. Markdown Enhancements

The notedown plugin lets you create Jupyter notes with Markdown, and also supports creating and editing Markdown files online.

Install

1
pip install https://github.com/aaren/notedown/tarball/master

Configure

Edit ~/.jupyter/jupyter_notebook_config.py and add the following content:

1
c.NotebookApp.contents_manager_class = 'notedown.NotedownContentsManager'

6. References

https://github.com/gopherdata/gophernotes