Tooling

The Pumas suite is agnostic to the tooling that you use to run your code and prepare your reports. For example, it is possible to run Julia and Pumas in a plain terminal, an IDE like VSCode, a cloud platform like JuliaHub, a publishing system like Quarto, or any combination of those. This page lists some tooling options which have been a good match with Pumas in our experience, or for which we have developed additional functionality.

Quarto

Quarto is a publishing system based on pandoc. Users write markdown-based notebooks with executable code blocks embedded in them. During the rendering step, these code blocks are then run and their outputs like figures and tables spliced into the result. This workflow allows to target a wide range of possible output formats such as Word documents, PDFs, HTML pages and more with a single source file.

Since prerelease version 1.5.29, quarto has support for a native Julia engine based on the QuartoNotebookRunner package. This engine can be selected by specifying the following frontmatter:

---
engine: julia
---

More information about the julia engine can be found in the official documentation.

QuartoTools

QuartoTools is a package that contains helper functions for use with quarto's native julia engine.

Serialization

When working with serialized data in Quarto notebooks users must use the QuartoTools.serialize and QuartoTools.deserialize functions provided by the QuartoTools package rather than the Serialization package. This is due to the differences in the behaviour of code evaluation between the Julia REPL and that of Quarto. These two functions are drop-in replacements for those provided by Serialization and fall back on the implementation provided by it when not run in a Quarto notebook. This means that simply replacing using Serialization with using QuartoTools should be sufficient to allow for transparent serialization and deserialization between notebooks, batch scripts, and the REPL.

Note that if both QuartoTools and Serialization are imported with using in the same session then the functions serialize and deserialize will need to be prefixed with their package name due to the name collisions between the two packages. Typically users should only need to import QuartoTools.

Cell expansion

QuartoNotebookRunner has a special feature called cell expansion. It allows to have a single code cell that outputs what looks like multiple code cells and their outputs to quarto.

What can you use this feature for?

Quarto has many advanced options which allow you to create richer output, for example tabsets which can group several separate sections of a quarto notebook into selectable tabs. These features are controlled with markdown annotations, for example, a tabset follows this structure:

::: {.panel-tabset}

## Tab 1

Content of tab 1

## Tab 2

Content of tab 2

... possibly more tabs ...

:::

As you can see, the tabset begins and ends with a pandoc ::: div fence and consists of sections demarcated by markdown headings. This mechanism has two drawbacks for the user:

  • It can be tricky to get the syntax right, especially with multiple nested ::: fences that need to be closed correctly. (This also applies when you generate markdown programmatically by printing out snippets in loops and using the output: asis cell option.)
  • It is static. Each tab has to be written into the source markdown explicitly, so you cannot easily create a tabset with a dynamic number of tabs. For example, a tabset with one plot per tab where the number of plots depends on runtime information and therefore is not known in advance.

Cell expansion can solve both of these problems. As expand can return an arbitrary number of QuartoNotebookWorker.Cells, we can use these cells to build the structures quarto expects programmatically, instead of having to hardcode them into the notebook.

For example, a tabset with plots could be generated by expanding into:

  • a cell with markdown output ::: {.panel-tabset}
  • a cell with markdown output ## Tab 1
  • a cell with a plot output, for example a Makie.Figure
  • more headings and plots
  • a cell with markdown output :::
Note

Cell expansion is not code generation. We do not generate and evaluate arbitrary code. Instead, we create objects describing code cells together with their outputs which is easier to reason about and more composable.

Each QuartoNotebookWorker.Cell has three fields:

  • thunk stores a function which returns the fake cell's output value when run. This value is treated as any other code cell output value, so it may be of any type that the display system can handle, and it may even be expandable itself (allowing for recursive expansion).
  • code may hold a string which will be rendered as the code of the fake cell (this code is not run)
  • options is a dictionary of quarto cell options, for example "echo" => false to hide the source code section

QuartoTools defines a set of helper objects that can serve as building blocks that can be composed further. For example, a Tabset may contain multiple Divs, each describing a two-column layout which is populated with two plots.

QuartoTools.serialize — Function
serialize(s::IO, x)
serialize(filename::AbstractString, x)

Serialize x to the given IO stream or file using Julia's built-in serialization while correctly handling differences in "root" evaluation module between the REPL and Quarto notebooks.

QuartoTools.deserialize — Function
deserialize(s::IO)
deserialize(filename::AbstractString)

Deserialize a value from the given IO stream or file using Julia's built-in serialization while correctly handling differences in "root" evaluation module between the REPL and Quarto notebooks.

QuartoTools.Cell — Type
struct Cell

Cell(content::Function; code = nothing, options = Dict{String,Any}(), lazy = true)
Cell(content; code = nothing, options = Dict{String,Any}(), lazy = false)

The most basic expandable object, representing a single code cell with output.

If code === nothing, the code cell will be hidden by default using the quarto option echo: false. Note that code is never evaluated, merely displayed in code cell style. Only content determines the actual cell output.

All options are written into the YAML options header of the code cell, this way you can use any cell option commonly available in code cells for your generated cells. For example, options = Dict("echo" => false) will splice #| echo: false into the code cell's options header.

If lazy === true, the output will be treated as a thunk, which has to be executed by QuartoNotebookRunner to get the actual output object that should have display called on it. Accordingly, you will get an error if the output object is not a Base.Callable. If lazy === false, the output will be used as the actual output object directly by QuartoNotebookRunner. As an example, if you generate a hundred plot output cells, it is probably better to generate the plots using lazy functions, rather than storing all of them in memory at once. The lazy option is set to true by default when a Function is passed to the convenience constructor, and to false otherwise.

QuartoTools.Div — Type
struct Div

Div(children::Vector; id=[], class=[], attributes=Dict())
Div(child; kwargs...)

Construct a Div which is an expandable that wraps its child cells with two markdown fence cells to create a pandoc div using ::: as the fence delimiters. Div optionally allows to specify one or more ids, classes and key-value attributes for the div.

id and class should each be either one AbstractString or an AbstractVector of those. attributes should be convertible to a Dict{String,String}.

Examples

Div(Cell(123))
Div(
    [Cell(123), Cell("ABC")];
    id = "someid",
    class = ["classA", "classB"],
    attributes = Dict("somekey" => "somevalue"),
)
QuartoTools.Expand — Type
struct Expand

Expand(expandables::AbstractVector)

Construct an Expand which is an expandable that wraps a vector of other expandable. This allows to create multiple output cells using a single return value in an expanded quarto cell.

Example

Expand([Cell(123), Cell("ABC")])
QuartoTools.Tabset — Type
struct Tabset

Tabset(pairs; group = nothing)

Construct a Tabset which is an expandable that expands into multiple cells representing one quarto tabset (using the ::: {.panel-tabset} syntax).

pairs should be convertible to a Vector{Pair{String,Any}}. Each Pair in pairs describes one tab in the tabset. The first element in the pair is its title and the second element its content.

You can optionally pass some group id as a String to the group keyword which enables quarto's grouped tabset feature where multiple tabsets with the same id are switched together.

Example

Tabset([
    "Tab 1" => Cell(123),
    "Tab 2" => Cell("ABC")
])
QuartoTools.MarkdownCell — Function
MarkdownCell(s::String)

A convenience function which constructs a Cell that will be rendered by quarto with the output: asis option. The string s will be interpreted as markdown syntax, so the output will look as if s had been written into the quarto notebook's markdown source directly.