Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Jupyter Enhancement Proposal

Background

Project Jupyter has an existing repository and process for “Jupyter Enhancement Proposals” or JEP for short. The repository is here:

https://github.com/jupyter/enhancement-proposals

This format was originally used by IPython (IPEP) and continues to be used by the Python Core language (PEP). As currently envisioned, JEPs are written documents, in the form of a Pull Request to this repo, which describes significant proposed units of work on the projects. The README of the repo summarizes this as:

“Jupyter Enhancement Proposals will be used when presenting changes or additions that affect multiple components of the Jupyter ecosystem OR changes to a single key component.”

Since the JEP process was created, Jupyter has grown in the number of users, diversity of usage cases, number of contributors, and number of repos/org. For a variety of reasons, the JEP process has not scaled with this growth. The current situation is characterized by:

  1. Given the scope of development, it is difficult to follow all of the activity;

  2. As a result, most decisions are made in a distributed manner by local teams of contributors working on particular repos and orgs (hub, lab, ipython, jupyter-widgets, etc.); as a consequence:

  3. Distributed decisions that affect the project as a whole are made, without all relevant parties being able to participate and contribute.

  4. This is leading to fragmentation and slow progress in project wide concerns, such as the core protocols and standards of Jupyter.

As a result, there is a need to revitalize and reorganize an efficient and robust centralized proposal and decision making process for major work units across the project.

Goals and Tenets

Project Jupyter has adopted the Jupyter Enhancement Proposal process (JEP) to address distributed collaboration and experimentation as the project scales in the dimensions of contributors, components, and lines of code. At a high level, the primary guiding principle of the JEP process is to encourage collaboration and discussion as early as possible in the lifecycle of a major proposed change in Jupyter, with the goal of preventing costly rework, competing ideas, and unnecessary forking or fragmentation of ideas.

Several sub-goals exist for this process:

With that in mind, the JEP process operates under the following tenets:

JEP Submission Workflow

Phase 1: Pre-proposal

This is the least formal stage of any jupyter enhancement proposals. During this phase, discussions on the mailing list, in-person, on github issues are all fine to gauge community interest, feasibility, consequences, and to scope out technical impact and implementation details.

In order to transition out of the pre-proposal stage, the following checklist must be complete:

  1. A github issue on the Jupyter Enhancement Proposals repo is created that

    1. Briefly outlines the proposal

    2. Suggested review team (optional)

    3. Why it should be a JEP

      1. See the “JEP / Not-a-JEP Rubric” below.

  3. A Shepherd is identified to see the process through. (Shepherds are assigned on a round-robin basis from a set of interested engaged volunteers).

  4. A number is assigned to the JEP to track it through the rest of the process.

Outcome:

The Shepherd decides if the JEP criteria have been met.

Phase 2: RFC for the JEP

Submission: The author submits an initial draft of the JEP as a PR to the JEP repository starting from the relevant template decided in the pre-proposal stage. The Shepherd assigns a number (the GitHub PR number? A sequential number? Perhaps the number isn’t assigned until the JEP is merged?). The Shepherd assigns a Review Team.

Request For Comment (RFC) phase: The proposal is iterated on with feedback from the Review Team and the community at large. The Shepherd helps engage the Review Team. After the Review Team members have signed off on the JEP, with the criteria that there are no major objections, and at least some of the Review Team are in favor, the Shepherd initiates the Final Comment Period.

Final Comment Period (FCP): The community at large has 10 calendar days in which to provide any final comments on the JEP. A JEP may revert back to RFC if objections are supported by the Review Team. If not reverted to the RFC phase, the JEP is approved at the end of the FCP.

Phase 3: Work Commences

Once a JEP has been merged into the jupyter/enhancement-proposal repository, development work can commence on the JEP. As the implementer(s) is submitting a pull request or pull requests in relation to the JEP, they should provide a reference to the JEP so that reviewer has background context on the JEP.

As the JEP is being implemented, the implementer(s) are submitting pull requests for the JEP, they should update the JEP with addendums to denote the progress of the implementation using the following stages.

  1. In progress implementation via (list of PRs).

  2. Fully implemented via (list of PRs).

If in the course of implementation, it is discovered that the implementation needs to be radically different from what was defined in the original JEP, then a pull request needs to be submitted to modify the original JEP with the new necessary implementation and a note citing the need for a modification to the JEP. This pull request should be re-approved by the original review team.

If in the course of the implementation, the implementer(s) can choose to withdraw from the original JEP if they are no longer interested in implementing the JEP or see infeasibilities in the JEP.

Paths of the status of JEPs

img

What qualifies as a JEP?

This section contains a set of principles to help determine when something is a JEP. The principles will be used to determine when something becomes a PR during the JEP pre-proposal stage, as well as to determine when a PR becomes a JEP at an individual repo level.

Principles to follow

Below are a few example guidelines to follow when deciding if an idea should include a JEP (If yes, it requires a JEP). Under each question is a relevant example proposal.

Distribution

This section describes how information about the JEP process (e.g., new JEPs, updates to current JEPs, etc) is communicated to the community.

Note: This JEP repo is the canonical “source of truth” for individual JEPs, the JEP process, and activity on JEPs.

The JEP public archive website

A public website contains a readable archive of all JEP proposals. It contains list of all JEPs that have entered a “final” state (e.g., “Completed”, “Withdrawn”, “Rejected”). The content of each JEP will be displayed in a readable fashion. When a JEP enters into a final state, it is added to this website.

Note that the JEPs themselves contain the content, while the website is just a quick way to display them in a reading-friendly format.

Glossary

JEPs
Standalone Jupyter server
JEPs
Voilà Incorporation