Tooling
The Pumas suite is agnostic to the tooling that you use to run your code and prepare your reports. For example, it is possible to run Julia and Pumas in a plain terminal, an IDE like VSCode, a cloud platform like JuliaHub, a publishing system like Quarto, or any combination of those. This page lists some tooling options which have been a good match with Pumas in our experience, or for which we have developed additional functionality.
Quarto
Quarto is a publishing system based on pandoc. Users write markdown-based notebooks with executable code blocks embedded in them. During the rendering step, these code blocks are then run and their outputs like figures and tables spliced into the result. This workflow allows to target a wide range of possible output formats such as Word documents, PDFs, HTML pages and more with a single source file.
Since prerelease version 1.5.29, quarto has support for a native Julia engine based on the QuartoNotebookRunner package. This engine can be selected by specifying the following frontmatter:
---
engine: julia
---
More information about the julia engine can be found in the official documentation.
QuartoTools
QuartoTools is a package that contains helper functions for use with quarto's native julia engine.
Serialization
When working with serialized data in Quarto notebooks users must use the QuartoTools.serialize
and QuartoTools.deserialize
functions provided by the QuartoTools
package rather than the Serialization
package. This is due to the differences in the behaviour of code evaluation between the Julia REPL and that of Quarto. These two functions are drop-in replacements for those provided by Serialization
and fall back on the implementation provided by it when not run in a Quarto notebook. This means that simply replacing using Serialization
with using QuartoTools
should be sufficient to allow for transparent serialization and deserialization between notebooks, batch scripts, and the REPL.
Note that if both QuartoTools
and Serialization
are imported with using
in the same session then the functions serialize
and deserialize
will need to be prefixed with their package name due to the name collisions between the two packages. Typically users should only need to import QuartoTools
.
Cell expansion
QuartoNotebookRunner has a special feature called cell expansion. It allows to have a single code cell that outputs what looks like multiple code cells and their outputs to quarto.
What can you use this feature for?
Quarto has many advanced options which allow you to create richer output, for example tabsets which can group several separate sections of a quarto notebook into selectable tabs. These features are controlled with markdown annotations, for example, a tabset follows this structure:
::: {.panel-tabset}
## Tab 1
Content of tab 1
## Tab 2
Content of tab 2
... possibly more tabs ...
:::
As you can see, the tabset begins and ends with a pandoc :::
div fence and consists of sections demarcated by markdown headings. This mechanism has two drawbacks for the user:
- It can be tricky to get the syntax right, especially with multiple nested
:::
fences that need to be closed correctly. (This also applies when you generate markdown programmatically by printing out snippets in loops and using theoutput: asis
cell option.) - It is static. Each tab has to be written into the source markdown explicitly, so you cannot easily create a tabset with a dynamic number of tabs. For example, a tabset with one plot per tab where the number of plots depends on runtime information and therefore is not known in advance.
Cell expansion can solve both of these problems. As expand
can return an arbitrary number of QuartoNotebookWorker.Cell
s, we can use these cells to build the structures quarto expects programmatically, instead of having to hardcode them into the notebook.
For example, a tabset with plots could be generated by expanding into:
- a cell with markdown output
::: {.panel-tabset}
- a cell with markdown output
## Tab 1
- a cell with a plot output, for example a
Makie.Figure
- more headings and plots
- a cell with markdown output
:::
Cell expansion is not code generation. We do not generate and evaluate arbitrary code. Instead, we create objects describing code cells together with their outputs which is easier to reason about and more composable.
Each QuartoNotebookWorker.Cell
has three fields:
thunk
stores a function which returns the fake cell's output value when run. This value is treated as any other code cell output value, so it may be of any type that the display system can handle, and it may even be expandable itself (allowing for recursive expansion).code
may hold a string which will be rendered as the code of the fake cell (this code is not run)options
is a dictionary of quarto cell options, for example"echo" => false
to hide the source code section
QuartoTools defines a set of helper objects that can serve as building blocks that can be composed further. For example, a Tabset
may contain multiple Div
s, each describing a two-column layout which is populated with two plots.
QuartoTools.serialize
— Functionserialize(s::IO, x)
serialize(filename::AbstractString, x)
Serialize x
to the given IO stream or file using Julia's built-in serialization while correctly handling differences in "root" evaluation module between the REPL and Quarto notebooks.
QuartoTools.deserialize
— Functiondeserialize(s::IO)
deserialize(filename::AbstractString)
Deserialize a value from the given IO stream or file using Julia's built-in serialization while correctly handling differences in "root" evaluation module between the REPL and Quarto notebooks.
QuartoTools.Cell
— Typestruct Cell
Cell(content::Function; code = nothing, options = Dict{String,Any}(), lazy = true)
Cell(content; code = nothing, options = Dict{String,Any}(), lazy = false)
The most basic expandable object, representing a single code cell with output.
If code === nothing
, the code cell will be hidden by default using the quarto option echo: false
. Note that code
is never evaluated, merely displayed in code cell style. Only content
determines the actual cell output.
All options
are written into the YAML options header of the code cell, this way you can use any cell option commonly available in code cells for your generated cells. For example, options = Dict("echo" => false)
will splice #| echo: false
into the code cell's options header.
If lazy === true
, the output
will be treated as a thunk, which has to be executed by QuartoNotebookRunner to get the actual output object that should have display
called on it. Accordingly, you will get an error if the output
object is not a Base.Callable
. If lazy === false
, the output
will be used as the actual output object directly by QuartoNotebookRunner. As an example, if you generate a hundred plot output cells, it is probably better to generate the plots using lazy functions, rather than storing all of them in memory at once. The lazy
option is set to true
by default when a Function
is passed to the convenience constructor, and to false
otherwise.
QuartoTools.Div
— Typestruct Div
Div(children::Vector; id=[], class=[], attributes=Dict())
Div(child; kwargs...)
Construct a Div
which is an expandable that wraps its child cells with two markdown fence cells to create a pandoc div using :::
as the fence delimiters. Div
optionally allows to specify one or more ids, classes and key-value attributes for the div.
id
and class
should each be either one AbstractString
or an AbstractVector
of those. attributes
should be convertible to a Dict{String,String}
.
Examples
Div(Cell(123))
Div(
[Cell(123), Cell("ABC")];
id = "someid",
class = ["classA", "classB"],
attributes = Dict("somekey" => "somevalue"),
)
QuartoTools.Expand
— Typestruct Expand
Expand(expandables::AbstractVector)
Construct an Expand
which is an expandable that wraps a vector of other expandable. This allows to create multiple output cells using a single return value in an expanded quarto cell.
Example
Expand([Cell(123), Cell("ABC")])
QuartoTools.Tabset
— Typestruct Tabset
Tabset(pairs; group = nothing)
Construct a Tabset
which is an expandable that expands into multiple cells representing one quarto tabset (using the ::: {.panel-tabset}
syntax).
pairs
should be convertible to a Vector{Pair{String,Any}}
. Each Pair
in pairs
describes one tab in the tabset. The first element in the pair is its title and the second element its content.
You can optionally pass some group id as a String
to the group
keyword which enables quarto's grouped tabset feature where multiple tabsets with the same id are switched together.
Example
Tabset([
"Tab 1" => Cell(123),
"Tab 2" => Cell("ABC")
])
QuartoTools.MarkdownCell
— FunctionMarkdownCell(s::String)
A convenience function which constructs a Cell
that will be rendered by quarto with the output: asis
option. The string s
will be interpreted as markdown syntax, so the output will look as if s
had been written into the quarto notebook's markdown source directly.