PDF Reports

Standardized PDF reports can be generated from analysis results. These combine all the plots described in the Plotting section of the manual into a single standalone PDF document along with several tables. Reports can be generated for both Pumas and NCA results. Report generation is accessed via the report function.

Pumas

For Pumas fitted models we can generate reports with equivalent content to that which is presented in the evaluate_diagnostics interactive application using the same input syntax for the fitted models. Please see the section of supported syntax for evaluate_diagnostics.

Note

The report function discussed in this section only supports continuous data models. Support for discrete data models will be made available soon.

For a single model report based on a fitted model called my_fitted_model we can use the following:

report(my_fitted_model)

Once the above report call returns you will find in the output folder a PDF report containing the following sections:

  • a cover page
  • table of contents
  • list of figures
  • list of tables

For each fitted model the following is also included:

  • listing of metadata associated
  • latexify output for each block in the model
  • metric and coefficient tables
  • data checkout plots for the populations
  • goodness-of-fit plots
  • visual predictive check plots if any VPCs are provided
  • distribution plots
  • convergence trace plots
  • tables of individual coefficients
  • complete printout of logged output from fitting
  • details of the Manifest.toml associated with the active Julia project

Pumas Customization

The following attributes can be customized on a per-report basis. Each one is provided as a keyword argument to report:

PlottingUtilities.report — Function
report(
    fitted_models;
    output,
    title,
    version,
    date,
    author,
    categorical,
    force,
    inspect,
    infer,
    clean,
    header,
    footer,
    plot_fontsize,
    plot_resolution
)

Create a PDF report based on the provided fitted_models.

fitted_models can be provided as one of several different inputs:

A single fitted model result from fit.

fit_result = fit(...)
report(fit_result)

Note that ... in these examples is not real syntax and is used in these examples to illustrate unimportant details.

A vector of fitted model results from fit.

fit_1 = fit(...)
fit_2 = fit(...)
report([fit_1, fit_2])

The above two calls will result in fitted models named 1 to N. If you wish to provide named fitted models in the resulting report then pass in a NamedTuple instead of a single fit result or vector:

my_named_fit = fit(...)
report((; my_named_fit))
fit_1 = fit(...)
fit_2 = fit(...)
report((; fit_1, fit_2))

Note the leading ;, which is standard Julia syntax and required for naming your fitted models.

Other computed results can also be included alongside your fit results, namely inspect, infer, and vpc results. Any number of vpc results can be included, while only one of each of inspect and infer may be included. To include associated results along with the fitted model use Julia's tuple syntax, i.e. (a, b, c):

fit_1 = fit(...)
ins_1 = inspect(fit_1, ...)
inf_1 = infer(fit_1, ...)
vpc_1 = vpc(fit_1, ...)
vpc_2 = vpc(fit_1, ...)

report((; my_model = (fit_1, ins_1, inf_1, vpc_1, vpc_2)))

Keywords

  • output: directory into which the final report files will be saved.
  • title: a title for the final report that will appear on the title page and be used for the name of the finished PDF.
  • version: the version of the document, given as a v"..." string.
  • date: the date of the report, given as a Date value.
  • author: the name of the author of the document.
  • categorical: a vector of Symbols for covariates that should be treated as categorical in all plots of the report.
  • force: (default true) remove all contents of output (if it exists) before generating a new report.
  • inspect: (default true) should inspect be called automatically on fitted models if no inspect result is already provided. The call to inspect does not pass any additional arguments and so if customized inspect calls are required, for example when using NPDE, a pre-computed inspect result should be passed in instead of relying of the automatic behaviour.
  • infer: (default false) should infer be called automatically on fitted models if no infer result is already provided.
  • clean: (default true) only include the final PDF document in the output folder rather than all intermediate files as well.
  • header: custom text to be added to the header section of each page.
  • footer: custom text to be added to the footer section of each page.
  • plot_fontsize: set the font size for all plots in the report. Default is 16.
  • plot_resolution: set the resolution for all plots in the report. Default is (960, 540).

Examples

The following will generate a report based on objects fpm1 and fpm2, which are FittedPumasModels. It will automatically call inspect and infer for each.

report((; fpm1, fpm2))

The names of the models in the reports will be the same as the variable names. To change the names pass a different key name in the NamedTuple syntax passed to report.

Report-level metadata that should be included in the generated report can be passed in via the keyword arguments title and version, others to be added later.

The name of the resulting PDF file can be given with output = "NAME.pdf.

report(figures::Vector{Figure})

Create a report document containing all the figures in figures provided in figures.

Keywords

  • output the directory in which the final document is saved.
  • title required, the title of the document.
  • author the author of the document.
  • header contents of the page headers.
  • footer contents of the page footers.
  • caption caption to use for each figure.
  • force replace the output directory if it exists.
  • clean only save the final document, not intermediate files.
  • landscape use landscape orientation for the figures in the document.
  • logo path to a png or svg file used at the bottom of the front page.
  • logo_width_pt width in pt of the logo on the bottom of the front page.

NCA

For NCA we can generate reports for NCAPopulation and NCAReport data. The following input variations are supported for report for NCA. ... syntax here is not legal Julia syntax and is only used to represent details unimportant to the given examples.

Generate a report using all defaults from a run_nca result:

nca_report = run_nca(...)
report(nca_report)

A report using a customized summarize rather than the default:

nca_report = run_nca(...)
nca_summary = summarize(...)
report(nca_report, nca_summary)

Or by using a NCAPopulation directly and specifying all the required keywords directly to report rather than the intermediate run_nca and summarize functions.

report(
    nca_population;
    parameters = ...,
    stratify_by = ...,
    studytitle = ...,
    sigdigits = ...,
)

Once the report call has finished you will find in the output/ folder a PDF report containing the following sections:

  • a cover page
  • table of contents
  • list of figures
  • list of tables
  • report metadata passing in via keywords or the NCAReport object
  • a table of summary statistics
  • linear and log plots of observations summary
  • individual subject fit plots in linear and log scale

Customization

The following attributes can be customized on a per-report basis. Each one is provided as a keyword argument to report. In addition to the keywords provided in run_nca and summarize, report for NCAPopulation and NCAReport also supports the following keywords.

  • output::String specifies the folder, relative to the currently active directory, into which the resulting PDF report should be saved. If this folder already exists then it will be overwritten. Use force=false to disable this behavior.
  • force::Bool is used to either overwrite (true) the output folder prior to generating a new report, or to throw an error (false). The default is true, set to false if you do not want to automatically overwrite the output folder.
  • clean::Bool controls whether to only save the final PDF report to output, or to include all intermediate files as well. Set to clean = false if you would like to not clean the output folder. Defaults to true.
  • header::String and footer::String can be used to add customized headers and footers to every page of the report aside from the title page.

Pumas

Note

The report function discussed in this section only supports continuous data models. Support for discrete data models will be made available soon.

For a single model report based on a fitted model called my_fitted_model we can use the following:

report(my_fitted_model)

Once the above report call returns you will find in the output folder a PDF report containing the following sections:

  • a cover page
  • table of contents
  • list of figures
  • list of tables

For each fitted model the following is also included:

  • listing of metadata associated
  • latexify output for each block in the model
  • metric and coefficient tables
  • data checkout plots for the populations
  • goodness-of-fit plots
  • visual predictive check plots if any VPCs are provided
  • distribution plots
  • convergence trace plots
  • tables of individual coefficients
  • complete printout of logged output from fitting
  • details of the Manifest.toml associated with the active Julia project

Customization

The following attributes can be customized on a per-report basis. Each one is provided as a keyword argument to report.

  • output::String specifies the folder, relative to the currently active directory, into which the resulting PDF report should be saved. If this folder already exists then it will be overwritten. Use force = false to disable this behavior.
  • title::String specifies the report title to be used for both the title page of the document and the name of the final PDF file.
  • version::VersionNumber can be used to embed a version number of the metadata section of the PDF document.
  • date::Dates.Date specifies the date to be used on the title page of the document. It will default to the current date if not provided.
  • author::String allows the author name of the report to be included on the title page.
  • categorical::Vector{Symbol} a vector of covariate names in the fitted models that should be treated as categorical when generating plots.
  • force::Bool is used to either overwrite (true) the output folder prior to generating a new report, or to throw an error (false). The default is true, set to false if you do not want to automatically overwrite the output folder.
  • inspect::Bool can be used to control whether Pumas.inspect is called on each fitted model if no Pumas.inspect result can be found in the provided inputs. This defaults to true.
  • infer::Bool can be used to control whether Pumas.infer is called on each fitted model if no Pumas.infer result can be found in the provided inputs. This defaults to false.
  • clean::Bool controls whether to only save the final PDF report to output, or to include all intermediate files as well. Set to clean = false if you would like to not clean the output folder. Defaults to true.
  • header::String and footer::String can be used to add customized headers and footers to every page of the report aside from the title page.
PlottingUtilities.report — Function
report(
    fitted_models;
    output,
    title,
    version,
    date,
    author,
    categorical,
    force,
    inspect,
    infer,
    clean,
    header,
    footer,
    plot_fontsize,
    plot_resolution
)

Create a PDF report based on the provided fitted_models.

fitted_models can be provided as one of several different inputs:

A single fitted model result from fit.

fit_result = fit(...)
report(fit_result)

Note that ... in these examples is not real syntax and is used in these examples to illustrate unimportant details.

A vector of fitted model results from fit.

fit_1 = fit(...)
fit_2 = fit(...)
report([fit_1, fit_2])

The above two calls will result in fitted models named 1 to N. If you wish to provide named fitted models in the resulting report then pass in a NamedTuple instead of a single fit result or vector:

my_named_fit = fit(...)
report((; my_named_fit))
fit_1 = fit(...)
fit_2 = fit(...)
report((; fit_1, fit_2))

Note the leading ;, which is standard Julia syntax and required for naming your fitted models.

Other computed results can also be included alongside your fit results, namely inspect, infer, and vpc results. Any number of vpc results can be included, while only one of each of inspect and infer may be included. To include associated results along with the fitted model use Julia's tuple syntax, i.e. (a, b, c):

fit_1 = fit(...)
ins_1 = inspect(fit_1, ...)
inf_1 = infer(fit_1, ...)
vpc_1 = vpc(fit_1, ...)
vpc_2 = vpc(fit_1, ...)

report((; my_model = (fit_1, ins_1, inf_1, vpc_1, vpc_2)))

Keywords

  • output: directory into which the final report files will be saved.
  • title: a title for the final report that will appear on the title page and be used for the name of the finished PDF.
  • version: the version of the document, given as a v"..." string.
  • date: the date of the report, given as a Date value.
  • author: the name of the author of the document.
  • categorical: a vector of Symbols for covariates that should be treated as categorical in all plots of the report.
  • force: (default true) remove all contents of output (if it exists) before generating a new report.
  • inspect: (default true) should inspect be called automatically on fitted models if no inspect result is already provided. The call to inspect does not pass any additional arguments and so if customized inspect calls are required, for example when using NPDE, a pre-computed inspect result should be passed in instead of relying of the automatic behaviour.
  • infer: (default false) should infer be called automatically on fitted models if no infer result is already provided.
  • clean: (default true) only include the final PDF document in the output folder rather than all intermediate files as well.
  • header: custom text to be added to the header section of each page.
  • footer: custom text to be added to the footer section of each page.
  • plot_fontsize: set the font size for all plots in the report. Default is 16.
  • plot_resolution: set the resolution for all plots in the report. Default is (960, 540).

Examples

The following will generate a report based on objects fpm1 and fpm2, which are FittedPumasModels. It will automatically call inspect and infer for each.

report((; fpm1, fpm2))

The names of the models in the reports will be the same as the variable names. To change the names pass a different key name in the NamedTuple syntax passed to report.

Report-level metadata that should be included in the generated report can be passed in via the keyword arguments title and version, others to be added later.

The name of the resulting PDF file can be given with output = "NAME.pdf.

report(figures::Vector{Figure})

Create a report document containing all the figures in figures provided in figures.

Keywords

  • output the directory in which the final document is saved.
  • title required, the title of the document.
  • author the author of the document.
  • header contents of the page headers.
  • footer contents of the page footers.
  • caption caption to use for each figure.
  • force replace the output directory if it exists.
  • clean only save the final document, not intermediate files.
  • landscape use landscape orientation for the figures in the document.
  • logo path to a png or svg file used at the bottom of the front page.
  • logo_width_pt width in pt of the logo on the bottom of the front page.

Further Customization

The report function leverages the Publish package for creation of the final PDF reports. Creation of completely customized reports (ones that cannot be achieved with the options outlined in the last section) falls outside the scope of report. Please refer directly to the Publish documentation should you wish to write more bespoke documents.