Jupyter Blog - Kevin Bates

On-demand Notebooks with JupyterHub, Jupyter Enterprise Gateway and Kubernetes

2018-10-16T15:03:00+00:00

Jupyter Notebook has become the “de facto” platform used by data scientists to build interactive applications and to tackle big data and AI problems.

With the increased adoption of Machine Learning and AI by enterprises, we have seen more and more requirement to build analytics platforms that provide on-demand notebooks for data scientists and data engineers in general.

This article describes how to deploy multiple components from the Jupyter Notebook stack to provide an on-demand analytics platform powered by JupyterHub and Jupyter Enterprise Gateway on a Kubernetes cluster.

Image 1 — Deployment Architecture

On-Demand Notebooks Infrastructure

Below are the main components we are going to use to build our solution, and its high-level description:

JupyterHub enables the creation of a multi-user Hub which spawns, manages, and proxies multiple instances of the single-user Jupyter Notebook server providing the ‘as a service’ feeling we are looking for.

Jupyter Enterprise Gateway provides optimal resource allocations by enabling kernels to be launched in its own pod enabling notebook pods to have minimal resources while kernel specific resources are allocated/deallocated accordingly to its lifecycle. It also enables the base image of the kernel to become a choice.

Kubernetes enables easy management of containerized applications and resources with the benefit of Elasticity and multiple other quality of services.

JupyterHub Deployment

JupyterHub is the entry point for our solution, it will manage user authorization and provisioning of individual Notebook servers for each user.

JupyterHub configuration is done via a config.yaml, and the following settings are required:

Enable custom notebook configuration (coming from the customized user image).

hub:
  extraConfig: |-
    config = '/etc/jupyter/jupyter_notebook_config.py'

Define the docker image to be used when instantiating the notebook server for each user

Define custom environment variables used to connect the Notebook server with Jupyter Enterprise Gateway to enable support for remote kernels

singleuser:
  image:
    name: elyra/nb2kg
    tag: dev
  storage:
    dynamic:
      storageClass: nfs-dynamic
  extraEnv:
    KG_URL: <FQDN of Gateway Endpoint>
    KG_HTTP_USER: jovyan
    KERNEL_USERNAME: jovyan
    KG_REQUEST_TIMEOUT: 60

The complete config.yaml would look like the one below:

hub:
  db:
    type: sqlite-memory
  extraConfig: |-
    config = '/etc/jupyter/jupyter_notebook_config.py'
    c.Spawner.cmd = ['jupyter-labhub']
proxy:
  secretToken: "xxx"

ingress:
  enabled: true
  hosts:
    - <FQDN Kubernetes Master>

singleuser:
  defaultUrl: "/lab"
  image:
    name: elyra/nb2kg
    tag: 2.0.0.dev0
  storage:
    dynamic:
      storageClass: nfs-dynamic
  extraEnv:
    KG_URL: <FQDN of Gateway Endpoint>
    KG_HTTP_USER: jovyan
    KERNEL_USERNAME: jovyan
    KG_REQUEST_TIMEOUT: 60

rbac:
  enabled: true

debug:
  enabled: true

Detailed deployment instructions for JupyterHub can be found at Zero to JupyterHub for Kubernetes, but the command below would deploy it into a Kubernetes environment.

helm upgrade --install --force hub jupyterhub/jupyterhub --namespace hub --version 0.7.0 --values jupyterhub-config.yaml

Custom JupyterHub user image

By default, JupyterHub would deploy a vanilla Notebook Server image which will require that all resources ever used by the image to be allocated when the Kubernetes image is instantiated.

Our custom image will enable kernels to be started in its own pod, promoting a better resource allocation as resources can be allocated and freed up as needed. This also gives us the flexibility of supporting different frameworks for different notebooks (e.g. a notebook using Python and TensorFlow, while another is using Python and Caffe2).

Dockerfile for elyra-nb2kg custom image:

FROM jupyterhub/k8s-singleuser-sample:0.7.0

# Do the pip installs as the unprivileged notebook user
USER $NB_USER

ADD jupyter_notebook_config.py /etc/jupyter/jupyter_notebook_config.py

# Install NB2KG
RUN pip install --upgrade nb2kg && \
    jupyter serverextension enable --py nb2kg --sys-prefix

Jupyter Notebook custom configuration to override Notebook handlers with the ones from NB2KG that will enable the notebook to connect with the Enterprise Gateway that enables remote kernels.

from jupyter_core.paths import jupyter_data_dir
import subprocess
import os
import errno
import stat

c = get_config()
c.NotebookApp.ip = '*'
c.NotebookApp.port = 8888
c.NotebookApp.open_browser = False

c.NotebookApp.session_manager_class = 'nb2kg.managers.SessionManager'
c.NotebookApp.kernel_manager_class = 'nb2kg.managers.RemoteKernelManager'
c.NotebookApp.kernel_spec_manager_class = 'nb2kg.managers.RemoteKernelSpecManager'

# https://github.com/jupyter/notebook/issues/3130
c.FileContentsManager.delete_to_trash = False

# Generate a self-signed certificate
if 'GEN_CERT' in os.environ:
    dir_name = jupyter_data_dir()
    pem_file = os.path.join(dir_name, 'notebook.pem')
    try:
        os.makedirs(dir_name)
    except OSError as exc:  # Python >2.5
        if exc.errno == errno.EEXIST and os.path.isdir(dir_name):
            pass
        else:
            raise
    # Generate a certificate if one doesn't exist on disk
    subprocess.check_call(['openssl', 'req', '-new',
                           '-newkey', 'rsa:2048',
                           '-days', '365',
                           '-nodes', '-x509',
                           '-subj', '/C=XX/ST=XX/L=XX/O=generated/CN=generated',
                           '-keyout', pem_file,
                           '-out', pem_file])
    # Restrict access to the file
    os.chmod(pem_file, stat.S_IRUSR | stat.S_IWUSR)
    c.NotebookApp.certfile = pem_file

Note that the document above was generated by jupyter notebook --generate-config and then updated with the required handlers override:

c.NotebookApp.session_manager_class = 'nb2kg.managers.SessionManager'
c.NotebookApp.kernel_manager_class = 'nb2kg.managers.RemoteKernelManager'
c.NotebookApp.kernel_spec_manager_class = 'nb2kg.managers.RemoteKernelSpecManager'

Jupyter Enterprise Gateway deployment

Jupyter Enterprise Gateway enables Jupyter Notebook to launch and manage remote kernels in a distributed cluster, including Kubernetes cluster.

Enterprise Gateway provides a Kubernetes deployment descriptor that makes it simple to deploy it on a Kubernetes environment with the command below:

kubectl apply -f https://raw.githubusercontent.com/jupyter-incubator/enterprise_gateway/master/etc/kubernetes/enterprise-gateway.yaml

We also recommend that the kernel images be downloaded on all nodes of the Kubernetes cluster to avoid delays/timeouts when launching kernels for the first time on these nodes.

docker pull elyra/enterprise-gateway:dev
docker pull elyra/kernel-py:dev
docker pull elyra/kernel-tf-py:dev
docker pull elyra/kernel-r:dev
docker pull elyra/kernel-scala:dev

Automated One-Click Deployment using Ansible

If you are eager to get started and try this in a few machines, we have published an ansible script that deploys the full set of components described above on vanilla RHEL machines/VMs.

ansible-playbook --verbose setup-kubernetes.yml -c paramiko -i hosts-fyre-kubernetes

Conclusion

Jupyter Enterprise Gateway provides remote kernel management to Jupyter Notebooks. In a JupyterHub/Kubernetes environment, it enables hub to launch tiny Jupyter Notebook pods and only allocate large kernel resources when these are created as independent pods. This approach also allows for easy sharing of expensive resources as GPUs, etc

Special Thanks

Special thanks to Erik Sundell and Min RK from the JupyterHub team for the support and initial discussions around JupyterHub.

Introducing Jupyter Enterprise Gateway

2018-09-17T20:19:00+00:00

Yesterday, the Jupyter Steering Council voted to make Jupyter Enterprise Gateway a top-level Jupyter Project. I want to thank everyone for their contributions so far — code from my teammates at IBM and the community in general; advice from the Jupyter development team and mentors; and questions, issues, and requirements from end users.

As we become an official Jupyter project, I would like to take the opportunity to give an update on the project’s progress during our incubation period.

What is Jupyter Enterprise Gateway?

Jupyter Enterprise Gateway enables Jupyter Notebook to launch remote kernels in a distributed cluster, including Apache Spark managed by YARN, IBM Spectrum Conductor or Kubernetes.

Although Enterprise Gateway is mostly kernel agnostic, it provides out of the box configuration examples for the following kernels:

Python using IPython kernel
R using IRkernel
Scala using Apache Toree kernel

Jupyter Enterprise Gateway does not manage multiple Jupyter Notebook deployments, for that you should look for JupyterHub. Having said that, Enterprise Gateway can enable JupyterHub to launch remote kernels as individual Kubernetes pods, providing better resource allocation and enabling better environment management as each pod can be based on different images (e.g. TensorFlow, Anaconda, etc)

Supported Platforms

Jupyter Enterprise Gateway currently enables remote kernels in the following platforms:

Distributed Kernels in Apache Spark

Jupyter Enterprise Gateway leverages different resource managers to enable distributed kernels in Apache Spark clusters. One example shown below describes kernels being launched in YARN cluster mode across all nodes of a cluster.

Jupyter Enterprise Gateway leverages Apache Spark resource managers to distribute kernels

Note that, Jupyter Enterprise Gateway also provides some other value-added capabilities such as enhanced security and multiuser support with user impersonation.

Jupyter Enterprise Gateway provides Enhanced Security and Multiuser support with user Impersonation

Distributed Kernels in Kubernetes

Jupyter Enterprise Gateway support for Kubernetes enables decoupling the Jupyter Notebook Server and its kernels into multiple pods. This enables running Notebook server pods with minimally necessary resources based on the workload being processed.

Jupyter Enterprise Gateway enable remote kernels on Kubernetes cluster

Jupyter Enterprise Gateway and JupyterHub

JupyterHub is a multi-user server that manages and proxies multiple instances of the single-user Jupyter notebook server. Particularly in a Kubernetes environment, Jupyter Enterprise Gateway can enable JupyterHub to launch remote kernels as individual Kubernetes pods, providing better resource allocation and enabling better environment management as each pod can be based on different images (e.g. TensorFlow, Anaconda, etc). This has proven to be very desired, particularly when working on Deep Learning related Notebooks.

JupyterHub and Jupyter Enterprise Gateway together in a Kubernetes cluster

Some project metrics

The following stats have been collected from the Jupyter Enterprise Gateway GitHub repository during the incubation period:

10 releases

12 individual contributors

90 Stars

34 Forks

Source code, documentation, and other community resources

The Jupyter Enterprise Gateway community provides multiple resources that both users and contributors can use:

Source Code available at GitHub
https://github.com/jupyter/enterprise_gateway

Documentation available at ReadTheDocs
http://jupyter-enterprise-gateway.readthedocs.io/en/latest/

Automated builds available at Travis.CI
https://travis-ci.org/jupyter/enterprise_gateway

Releases available at PyPi.org and Conda Forge
https://pypi.org/project/jupyter_enterprise_gateway/
https://github.com/conda-forge/jupyter_enterprise_gateway-feedstock

Related Docker Images available at Elyra organization at DockerHub
https://hub.docker.com/u/elyra/dashboard/

What’s next?

We are eager to build an even greater community around the project, and tailor the project roadmap based on community advise.

Currently, we are busy working on advancing our Kubernetes support and integration with JupyterHub.

As always, we welcome questions, comments, and suggestions from users and the community in general.