Jupyter Blog - Antoine Prouvost

Instantly view Parquet files in JupyterLab with Arbalister

2026-01-29T16:18:00+00:00

A colleague sends you a SQLite file, a Parquet dataset, or an Avro snapshot.
You need to explore the content, but you’re not sure what’s inside or even how to open it. You shouldn’t have to write code, craft SQL queries, or recall the syntax for read_parquet in a Python library. Instead, you should be able to click the file once and instantly see the tables and metadata.

Despite being the de-facto standard for data science and exploration, JupyterLab lacks this basic capability out of the box. Of course, there are extremely well designed libraries to read, process, and transform tabular data — but opening and viewing a file with a table should be simpler. A workflow with libraries like Pandas or Polars require you to install relevant packages, create a notebook, and iterate on queries to get a full picture of the data.

import pandas as pd

pd.set_option("display.max_rows", 50)
pd.set_option("display.max_columns", 20)
pd.set_option("display.width", None)

df = pd.read_parquet("data/2025/PERFORMANCE_SNAPSHOT_2025_01_05.parquet")
df

Whether for a newcomer to data science, or someone that needs to investigate many different files with hard to remember schemas, this is a bit cumbersome. Comma separated value files (CSV) have better support: double click on the file to see it open (see figure below). That is a great way to understand the business logic and answer questions such as What type of information is in this file? What do values typically look like? Does it seem to contain all the data I need? What is the difference between customer_id and user_id?

The JupyterLab CSV viewer

Arbalister, our new JupyterLab extension, changes this. With Arbalister, you can double-click to instantly view a wide range of tabular data files: Parquet, CSV, Avro, ORC, SQLite, and more…

Arbalister viewer can open Parquet files and more

The extension will load data as it is displayed, making it possible to view it almost instantaneously even if the file does not fit in memory. A toolbar lets the user select some reading options, such as the delimiter for a CSV file, or selecting the table to display in the SQLite file!

Arrow over the wire

Fueling it all is the Apache Arrow ecosystem, an in-memory format for tabular data used in most data science libraries. Plenty of files types can already be read into an Arrow data structure so we exploit it heavily in Arbalister (with Apache Datafusion in our case).

The table displayed in the user’s browser reuses JupyterLab Lumino DataGrid with a custom data model. In the latter, we divide the whole table in chunks across both the rows and columns axes (e.g. 512 rows and 24 columns). When we need to display data from a chunk that is not already available, we make an HTTP request to a server-side extension that open the file, read the relevant portion into an Arrow table and return it to the client. The table is returned as Arrow IPC (Inter Process Communication, a binary format close to the original Arrow memory) in the response body. In the client, a lightweight Arrow implementation can read it back efficiently.

Arbalister client-server architecture

When nothing else is happening, the implementation will also pre-fetch some of the next chunks in the background so that scrolling past a chunk boundary is imperceptible.

Because it all works in Arrow, which is standard for tabular data, adding support for new types of files that can be read into the Arrow format is extremely easy. Vortex, Lance, or whichever other format becomes relevant. But that is not all…

The existing SQLite support is made possible through ADBC, an SQL connector that reads databases into Arrow format. SQLite databases fit into single files, which fit the abstraction used in Arbalister, but ADBC could also be used to add general database exploration (Postgres etc.) into Arbalister. With Datafusion, we could also add visualization for data lakehouses: partitioned datasets (e.g. DeltaLake or Apache Iceberg) over remote object storage (S3). With Intelligent predicate and projections pushdowns, we could visualize remote tera-byte table instantly by only downloading the relevant parts.

Looking further

There are multiple things that would make great improvements to Arbalister. We have already mentioned:

Optional support for newer file types: Vortex, Lance;
Object storage and data lakehouses support;
SQL databases support.

Another direction is support for JupyterLite, a WebAssembly distribution of JupyterLab running entirely in the browser. This setting is great for education because it does not require more than a static file server to host.
Being able to work directly with Parquet files in the browser is great for making dashboard and interactive demonstrations. Arrow and Datafusion already have some experimental support for WebAssembly so this is not so far-fetched.

Finally Arbalister could use a few features to investigate deeper questions about the data. For instance, beyond the What does the data look like? question, some column filters could help answer questions such as What does the data look like in November 2024 for John Doe? Perhaps even an SQL console for more advanced queries.

An extension point

The server extension being only a way to serve Arrow data, it can be reused by other libraries to use it in other ways. For instance, if we knew that our table contains time-series data, a second viewer could display it as a graph.
Similarly for geospatial data.

If you are interested to use or contribute to Arbalister, head to our GitHub!

About the Author

Antoine Prouvost is a senior scientific software engineer at QuantStack. His work on Arbalister was funded by Bloomberg.

A Jupyter kernel for GNU Octave

2023-01-11T17:33:00+00:00

GNU Octave is a long-standing member of the scientific computing ecosystem, featuring a mathematics-oriented syntax with built-in 2D/3D plotting and visualization tools, and compatible with the Matlab syntax. It is a natural candidate for high-quality integration in the Jupyter ecosystem.

Jupyter has been devised with language agnosticism in mind since it was spun off from the IPython project. The Kernel (the part of the infrastructure responsible for executing the code input by the user) is a key extension point to Jupyter, and dozens of language kernels exist in the ecosystem.

Prior art on Jupyter / GNU Octave integration includes the Calysto/Octave_kernel project by Steven Silvester (who is also a core Jupyter maintainer and the co-creator of JupyterLab). This existing kernel is part of the family of kernels built on top of the ipykernel reference implementation of the Jupyter protocol, in Python.

Today, we are happy to announce the xeus-octave project, a Jupyter kernel for GNU Octave. Xeus-octave was created by Giulio Girardi, recently joined by Antoine Prouvost — and has been incorporated into the Project Jupyter governance.

Xeus-octave is built upon the Xeus library, a C++ implementation of the Jupyter kernel protocol that enables fast development of new kernels using programming languages’ native APIs, unlocking new possibilities without parsing standard outputs or requiring a Python interpreter at runtime. After all, a Jupyter kernel is merely an executable providing a well-defined communication protocol. It is not bound to Python APIs.

A fully-featured kernel

We strove to make the xeus-octave kernel as complete as possible for this first iteration.

Multiple graphical toolkits

Notebook

A notebook Octave graphical toolkit can be used to render plots natively, and present the figure as an image in the notebook.

The classic Octave sombrero plot rendered as an image inside the notebook

Plotly

An experimental plotly Octave graphical toolkit can also use Plotly to render code in the browser, with dynamic views, zooms, and hover tooltips.

Multiple plots rendered in a single plotly figure

Rich display

The kernel binds with the C++ interface of GNU Octave to provide a complete Octave experience. Having access to the exact internal type of the objects, xeus-octave can present data with rich output.

Matrices can be presented as HTML or LaTeX tables.

Symbolic expressions can be presented as LaTeX formulas.

Structured data can be visualized as interactive tree views

Interactive help

Formatted help, extracted from the GNU Octave reference manual, can be queried using the ? syntax.

Trying it online

You can try the new Jupyter kernel for GNU Octave by clicking on the link below:

The Binder linked above includes examples of notebooks in the fields of Electronics, demonstrating the usability of xeus-octave in real-life complex scenarios.

This interactive demo is also featured on the Jupyter website at https://jupyter.org/try#kernels.

Installing xeus-octave

Xeus-octave has been packaged on conda-forge and can be installed with mamba or conda.

$ mamba install xeus-octave -c conda-forge

Future Work

In future iterations, we will work towards consolidating the xeus-octave experience, by adding cell magics and improving the Plotly integration.

A major ongoing development is the implementation of an Octave backend for Jupyter interactive widgets, based on the xwidgets project. Combined with Voilà, it will be possible to use GNU Octave to create standalone web dashboards.

Another ongoing endeavor is to port a growing portion of the scientific computing stack to WebAssembly (using the emscripten-forge project). Producing a Wasm build of GNU Octave will enable its use with Jupyterlite, enabling large-scale deployments on websites, blogs, and arbitrary web pages without any need for scalable cloud infrastructure.

About the authors

Antoine Prouvost is a Scientific Software Engineer at QuantStack and a xeus-octave maintainer. Previously, Antoine was researching Machine Learning and Combinatorial Optimization at the DS4DM research chair and Mila.

Giulio Girardi is a Master’s degree student in Electronic Engineering at the UNIPD, currently working as an Electronic Engineer at Protech Engineering. Giulio is the creator and a maintainer of the xeus-octave project.