-
Notifications
You must be signed in to change notification settings - Fork 16
DoodleBUGS: GSoC 2025 Report #126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 6 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
be34c47
Shravan's GSoC 2025 Report
shravanngoswamii 4a62ccd
update GSoC report
shravanngoswamii 5965b56
update team page
shravanngoswamii dc5afdb
minor css fix and report update
shravanngoswamii 15fb432
update report
shravanngoswamii f731f1f
update alias
shravanngoswamii c0920cf
update alias
shravanngoswamii 3b40276
Update news/posts/DoodleBUGS-Introduction/index.qmd
shravanngoswamii be14c87
Update index.qmd
shravanngoswamii bdd9437
Update index.qmd
shravanngoswamii 8bc454e
change folder name for report
shravanngoswamii 60170a9
Apply previous suggestion of @yebai
shravanngoswamii File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,252 @@ | ||
--- | ||
title: "Introducing DoodleBUGS: a Browser-Based Graphical Interface for JuliaBUGS" | ||
description: "Shravan Goswami's GSoC 2025 final report: goals, architecture, progress vs proposal, and how to try it." | ||
categories: | ||
- GSoC | ||
- Blog | ||
author: | ||
- name: Shravan Goswami | ||
url: https://shravangoswami.com/ | ||
date: 2025-08-28 | ||
aliases: | ||
- /news/posts/doodlebugs-gsoc-2025-final-report | ||
- /news/posts/doodlebugs-gsoc-2025 | ||
- /news/posts/doodlebugs-introduction | ||
- /news/posts/shravan-gsoc-2025-doodlebugs | ||
- /doodlebugs-gsoc-2025-final-report | ||
- /doodlebugs-gsoc-2025 | ||
- /doodlebugs-introduction | ||
- /shravan-gsoc-2025-doodlebugs | ||
- /news/posts/2025-08-28-DoodleBUGS-Introduction | ||
- /DoodleBUGS | ||
bibliography: references.bib | ||
csl: university-of-york-ieee.csl | ||
link-citations: true | ||
nocite: | | ||
@* | ||
--- | ||
|
||
## TL;DR | ||
|
||
- DoodleBUGS is a browser-based graphical editor for Bayesian models that utilize JuliaBUGS for BUGS model compilation and connect to Julia inference backends (e.g., AdvancedHMC via AbstractMCMC). | ||
shravanngoswamii marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
- Implemented: visual editor (nodes, edges, nested plates), legacy BUGS code generation that compiles with [JuliaBUGS](https://github.com/TuringLang/JuliaBUGS.jl) [@JuliaBUGS; @bugs-book], local execution via a Julia backend, unified standalone script generation (frontend), timeouts, multiple layouts, and extensive cleanup/typing. | ||
- Changed from proposal: frontend implemented in Vue 3 (instead of React); backend simplified (frontend is the single source of truth for standalone scripts). | ||
- Status: Working application. Live demo (static UI) available; for running inference locally, use the backend server. | ||
|
||
## Project Links | ||
|
||
- Repo: [https://github.com/TuringLang/JuliaBUGS.jl](https://github.com/TuringLang/JuliaBUGS.jl) | ||
- Live demo: [https://turinglang.org/JuliaBUGS.jl/DoodleBUGS/](https://turinglang.org/JuliaBUGS.jl/DoodleBUGS/) | ||
|
||
## DoodleBUGS Project Structure | ||
shravanngoswamii marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```{.bash} | ||
DoodleBUGS/ # Vite + Vue 3 app (UI editor) | ||
├── README.md # project documentation | ||
├── public/ # static assets served by Vite | ||
│ └── examples/ # example projects | ||
├── experiments/ # prototypes and exploratory work | ||
├── runtime/ # Julia HTTP backend (API endpoints & dependencies) | ||
├── src/ # application source | ||
│ ├── assets/ # styles and static assets | ||
│ ├── components/ # Vue components composing the UI | ||
│ │ ├── canvas/ # graph canvas and toolbars | ||
│ │ ├── common/ # shared UI primitives | ||
│ │ ├── layouts/ # app layout and modals | ||
│ │ │ └── MainLayout.vue # main application layout | ||
│ │ ├── left-sidebar/ # palette, project manager, execution settings | ||
│ │ ├── panels/ # code preview and data input panels | ||
│ │ ├── right-sidebar/ # execution, JSON editor, node properties | ||
│ │ └── ui/ # base UI elements (buttons, inputs, selects) | ||
│ ├── composables/ # reusable logic (codegen, drag & drop, graph, validator, grid) | ||
│ ├── config/ # configuration and node definitions | ||
│ ├── stores/ # Pinia state stores (graph, data, execution, project, UI) | ||
│ └── types/ # TypeScript types and ambient declarations | ||
├── tmp/ # local temporary outputs (ignored in builds) | ||
└── ztest/ # scratch/test artifacts | ||
``` | ||
|
||
## Motivation | ||
|
||
[JuliaBUGS](https://github.com/TuringLang/JuliaBUGS.jl) is a modern Julia implementation of the BUGS language [@bugs-rjournal; @bugs-book; @bugs-project]. DoodleBUGS revives the original visual modeling concept with a modern stack so users can: | ||
shravanngoswamii marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
- Construct probabilistic graphical models visually (nodes, edges, plates). | ||
- Export readable legacy BUGS code that compiles with JuliaBUGS [@JuliaBUGS; @bugs-rjournal; @bugs-book]. | ||
- Run inference and inspect results from the UI. Common BUGS applications include parallel MCMC [@multibugs], survival analysis [@bugs-survival], and Gibbs-style samplers [@albert-chib-1993; @informs-gibbs]. | ||
|
||
## What Was Built | ||
|
||
- Visual editor | ||
- Node types: stochastic, observed, deterministic | ||
- Plates with arbitrary nesting; robust drag-in/out and creation inside plates | ||
- Graph layouts: [WebCola](https://ialab.it.monash.edu/webcola/) and [ELK/KLay](https://www.eclipse.org/elk/); stable drag interactions | ||
- Legacy BUGS code generation [@bugs-rjournal; @bugs-book] | ||
- Topological ordering and plate-aware traversal | ||
- Parameter formatting and safe index expansion | ||
- Implemented in `DoodleBUGS/src/composables/useBugsCodeGenerator.ts` | ||
- Execution flow | ||
- Frontend sends `model_code`, `data`, `inits`, `settings` to backend | ||
- Backend compiles/samples and returns summaries and quantiles | ||
- Frontend unifies standalone script generation; backend no longer attaches duplicates | ||
shravanngoswamii marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
- Timeouts/resilience | ||
- Configurable timeout (frontend); enforced in backend worker | ||
- Safe temp directory cleanup on Windows with retries | ||
- Cleanup/typing | ||
- TypeScript fixes in `DoodleBUGS/src/components/right-sidebar/ExecutionPanel.vue` | ||
- Removal of unused backend code; consistent naming and logs | ||
|
||
## Architecture Overview | ||
|
||
- Frontend: [Vue 3](https://vuejs.org/), [Pinia](https://pinia.vuejs.org/), [Cytoscape.js](https://js.cytoscape.org/) [@cytoscapejs], [CodeMirror](https://codemirror.net/) | ||
- Code generation: `DoodleBUGS/src/composables/useBugsCodeGenerator.ts` | ||
- Execution panel: `DoodleBUGS/src/components/right-sidebar/ExecutionPanel.vue` | ||
- Backend (Julia) HTTP server | ||
- Server: `DoodleBUGS/runtime/server.jl` | ||
- Project deps: `DoodleBUGS/runtime/Project.toml` (HTTP, JSON3, JuliaBUGS, AbstractMCMC, AdvancedHMC, ReverseDiff, MCMCChains, DataFrames, StatsBase, Statistics) | ||
- Endpoints: GET `/api/health`; POST `/api/run` and `/api/run_model` | ||
- Execution: creates temp dir, writes `model.bugs` and `payload.json`, generates `run_script.jl`, enforces optional timeout | ||
|
||
## Design Principles and Architecture | ||
|
||
**Design principles** | ||
|
||
- Visual-first modeling with deterministic export to legacy BUGS [@bugs-rjournal; @bugs-book]. | ||
- Separation of concerns: editing (graph), generation (BUGS), execution (backend), and results (summary/quantiles) are modular. | ||
shravanngoswamii marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- Deterministic ordering: topological sort + plate-aware traversal ensures readable, stable code output. | ||
- Robustness: cancellable frontend fetch, backend-enforced timeout, and resilient temp cleanup on Windows (`safe_rmdir()`). | ||
|
||
**Frontend architecture (Vue 3 + Cytoscape.js)** | ||
|
||
- Core graph state is managed in Vue; [Cytoscape.js](https://js.cytoscape.org/) handles layout, hit-testing, and interaction semantics (including compound nodes for plates) [@cytoscapejs]. | ||
- Code generation lives in `DoodleBUGS/src/composables/useBugsCodeGenerator.ts` and maps `GraphNode`/`GraphEdge` to BUGS: | ||
- Kahn topological sort for definition order | ||
- Plate-aware recursion for `for (...) { ... }` blocks | ||
- Parameter canonicalization (indices, numeric/expr passthrough) | ||
- Standalone Julia script generation uses `generateStandaloneScript()` in the same composable, mirroring backend execution. | ||
|
||
**Backend architecture (Julia)** | ||
|
||
- `run_model_handler()` in `DoodleBUGS/runtime/server.jl` materializes `model.bugs`, `payload.json`, and a transient `run_script.jl` that: | ||
- Builds `NamedTuple`s from JSON or string-literal data/inits | ||
- Compiles via `JuliaBUGS.@bugs`, wraps with `ADgradient(:ReverseDiff)` [@ReverseDiff] | ||
- Samples with `AdvancedHMC.NUTS` through `AbstractMCMC` (Threads or Serial) [@AdvancedHMC; @AbstractMCMC; @HoffmanGelman2014] | ||
- Emits summaries (`MCMCChains`, `DataFrames`) and quantiles to JSON | ||
[@MCMCChains; @DataFrames] | ||
- Timeout: worker process is killed if exceeding `timeout_s`. | ||
- Cleanup: `safe_rmdir()` retries with GC to avoid EBUSY on Windows. | ||
|
||
## Why Vue (not React)? | ||
|
||
The proposal planned React; we chose Vue 3 after evaluating the graph layer and developer velocity for this app. | ||
|
||
- Tried Konva (canvas) for custom graph editing: powerful drawing primitives, but required bespoke graph semantics (hit testing, edge routing, compound nodes) that [Cytoscape.js](https://js.cytoscape.org/) provides out of the box. | ||
- Tried D3 force/layouts: flexible, but compound nodes (plates), nesting, and drag constraints became a significant amount of custom code to maintain. | ||
- [Cytoscape.js](https://js.cytoscape.org/) offered: | ||
- Native graph model with compound nodes (great for plates) | ||
- Integrated layouts (WebCola, KLay) and rich interaction APIs [@webcola; @elk] | ||
- Mature ecosystem and performance characteristics for medium-sized graphs | ||
- [Vue 3](https://vuejs.org/) (vs React) for this project: | ||
- Composition API made integrating an imperative graph library (Cytoscape) straightforward via composables and lifecycle hooks | ||
- SFC ergonomics and Pinia stores enabled quick iteration with strong TypeScript support | ||
- Template reactivity + refs reduced reconciliation overhead when bridging to Cytoscape’s imperative API | ||
- Minimal glue code for state management (Pinia) vs setting up reducers/selectors; enabled rapid iteration | ||
- Vite + Vue tooling yielded fast HMR for UI-heavy iterations | ||
- Design inspirations: draw.io for interaction affordances; Stan Playground for model/run UX [@drawio; @stan-playground]. | ||
|
||
## Comparison to Legacy DoodleBUGS | ||
shravanngoswamii marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The legacy tool was a desktop application driving WinBUGS [@winbugs]; the new DoodleBUGS is a browser-based editor targeting JuliaBUGS [@JuliaBUGS]. Key differences: | ||
|
||
- Platform and backend | ||
- Legacy: Desktop UI, WinBUGS execution pipeline | ||
- New: Web UI, Julia backend via `JuliaBUGS.@bugs`, sampling with `AdvancedHMC.NUTS` through `AbstractMCMC` | ||
- Graph engine and plates | ||
- Legacy: Bespoke graph handling with limited nesting semantics | ||
- New: [Cytoscape.js](https://js.cytoscape.org/) with compound nodes for robust nested plates; custom DnD for drag-in/out and creating inside plates | ||
- Layouts and interactions | ||
- Legacy: Limited auto-layout support | ||
- New: Multiple layout engines (Cola, Klay) and stable interactions; positions updated after `layoutstop` [@webcola; @elk] | ||
- Code generation | ||
- Legacy: Export to BUGS without strong ordering guarantees | ||
- New: Deterministic topological + plate-aware traversal; parameter canonicalization and safe index expansion | ||
- Execution and tooling | ||
- Legacy: WinBUGS-managed runs | ||
- New: Lightweight Julia HTTP backend, configurable timeouts, resilient temp cleanup, JSON summaries via `MCMCChains` | ||
- DevX and maintainability | ||
- New: Vue 3 + TypeScript + Pinia; unified standalone script generation on the frontend; leaner backend responses | ||
|
||
## Progress vs Proposal | ||
|
||
- Implemented | ||
- Visual editor with nested plates and robust DnD | ||
- BUGS code generator (topological + plate-aware) | ||
- Local execution + summaries/quantiles | ||
- Unified standalone script generation (frontend) | ||
- Timeouts/resilience | ||
- Multiple layouts and interactions | ||
- Extensive cleanup/typing | ||
- Execution timeout (end-to-end) | ||
- Layout options (Dagre (Hierarchical), fCoSE (Force-Directed), Cola (Physics Simulation), KLay (Layered)) and interactions | ||
- Cleanup and stronger typing | ||
- Changed | ||
- Vue 3 instead of React | ||
- Backend responses smaller; no standalone script attachment | ||
- Deferred/Partial | ||
- Rich diagnostics (R-hat, ESS, PPC, trace/density plots) | ||
- WebKit/Safari support | ||
- UX polish for large graphs | ||
|
||
## How to Run Locally | ||
|
||
Frontend (Vite): | ||
|
||
```bash | ||
# from repo root | ||
cd DoodleBUGS | ||
npm install | ||
npm run dev | ||
``` | ||
|
||
Backend (Julia): | ||
|
||
```bash | ||
# from repo root | ||
julia --project=DoodleBUGS/runtime DoodleBUGS/runtime/server.jl | ||
# server listens on http://localhost:8081 | ||
``` | ||
|
||
Notes: | ||
|
||
- CORS is enabled in the backend so the dev UI can call `http://localhost:8081`. | ||
- Live demo (static UI): https://turinglang.org/JuliaBUGS.jl/DoodleBUGS/ | ||
|
||
## API Summary | ||
|
||
- GET `/api/health` → `{ "status": "ok" }` | ||
- POST `/api/run` (alias: `/api/run_model`) | ||
- Body: `model_code`, `data`/`data_string`, `inits`/`inits_string`, `settings` `{ n_samples, n_adapts, n_chains, seed, timeout_s }` | ||
- Response: `{ success, summary, quantiles, logs, files[] }` | ||
|
||
See `DoodleBUGS/runtime/server.jl`. | ||
|
||
## Current Limitations | ||
|
||
- WebKit/Safari/iOS: unsupported at this time (see `DoodleBUGS/README.md`). | ||
- Limited visualization beyond summary/quantiles. | ||
- No persisted projects; session-based. | ||
|
||
## Future Work | ||
|
||
- Backend: Add Pluto.jl as a backend for supporting compound documents and QuartoNotebookRunner.jl for running notebooks. | ||
- Diagnostics/visualization: R-hat, ESS, trace plots, PPC, posterior densities | ||
- UX: richer node templates, validation, distribution hints | ||
- Persistence/sharing: save/load and shareable links | ||
- Browser compatibility: WebKit/Safari and iOS/iPadOS | ||
- Performance: virtualization for large graphs | ||
|
||
## Acknowledgements | ||
|
||
Much appreciation goes to my mentors Xianda Sun and Hong Ge. The work is impossible without your help and support. | ||
|
||
- Mentors: Xianda Sun ([\@sunxd3](https://github.com/sunxd3)) and Hong Ge ([\@yebai](https://github.com/yebai)) | ||
- TuringLang/JuliaBUGS community and contributors |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
@misc{bugs-rjournal, | ||
title = {The BUGS Language}, | ||
url = {https://journal.r-project.org/articles/RN-2006-005/RN-2006-005.pdf}, | ||
note = {R Journal/News article} | ||
} | ||
|
||
@misc{bugs-book, | ||
title = {The BUGS Book: A Practical Introduction to Bayesian Analysis}, | ||
url = {https://onlinelibrary.wiley.com/doi/10.1111/anzs.12058}, | ||
publisher = {Wiley} | ||
} | ||
|
||
@misc{bugs-project, | ||
title = {The BUGS Project}, | ||
url = {https://www.mrc-bsu.cam.ac.uk/software/bugs/} | ||
} | ||
|
||
@misc{multibugs, | ||
title = {MultiBUGS: Parallel BUGS Modeling}, | ||
url = {https://pmc.ncbi.nlm.nih.gov/articles/PMC7116196/} | ||
} | ||
|
||
@misc{bugs-survival, | ||
title = {Bayesian survival analysis with BUGS}, | ||
url = {https://onlinelibrary.wiley.com/doi/10.1002/sim.8933} | ||
} | ||
|
||
@misc{albert-chib-1993, | ||
title = {Inference via Gibbs (Albert \& Chib)}, | ||
url = {https://apps.olin.wustl.edu/faculty/chib/papers/albertchibjb93.pdf} | ||
} | ||
|
||
@misc{informs-gibbs, | ||
title = {Bayesian Inference Using Gibbs Sampling}, | ||
url = {https://pubsonline.informs.org/doi/10.1287/ited.2013.0120} | ||
} | ||
|
||
@article{HoffmanGelman2014, | ||
title = {The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo}, | ||
author = {Hoffman, Matthew D. and Gelman, Andrew}, | ||
year = {2014}, | ||
url = {https://arxiv.org/abs/1111.4246}, | ||
journal = {arXiv preprint arXiv:1111.4246} | ||
} | ||
|
||
@misc{AbstractMCMC, | ||
title = {AbstractMCMC.jl}, | ||
url = {https://github.com/TuringLang/AbstractMCMC.jl} | ||
} | ||
|
||
@misc{AdvancedHMC, | ||
title = {AdvancedHMC.jl}, | ||
url = {https://github.com/TuringLang/AdvancedHMC.jl} | ||
} | ||
|
||
@misc{ReverseDiff, | ||
title = {ReverseDiff.jl}, | ||
url = {https://github.com/JuliaDiff/ReverseDiff.jl} | ||
} | ||
|
||
@misc{MCMCChains, | ||
title = {MCMCChains.jl}, | ||
url = {https://github.com/TuringLang/MCMCChains.jl} | ||
} | ||
|
||
@misc{DataFrames, | ||
title = {DataFrames.jl}, | ||
url = {https://dataframes.juliadata.org/} | ||
} | ||
|
||
@misc{cytoscapejs, | ||
title = {Cytoscape.js}, | ||
url = {https://js.cytoscape.org/} | ||
} | ||
|
||
@misc{webcola, | ||
title = {WebCola}, | ||
url = {https://ialab.it.monash.edu/webcola/} | ||
} | ||
|
||
@misc{elk, | ||
title = {Eclipse Layout Kernel (ELK / KLay)}, | ||
url = {https://www.eclipse.org/elk/} | ||
} | ||
|
||
@misc{drawio, | ||
title = {draw.io (diagrams.net)}, | ||
url = {https://www.diagrams.net/} | ||
} | ||
|
||
@misc{stan-playground, | ||
title = {Stan Playground}, | ||
url = {https://stan-playground.flatironinstitute.org/} | ||
} | ||
|
||
@misc{winbugs, | ||
title = {WinBUGS}, | ||
url = {http://www.openbugs.net/w/FrontPage} | ||
} | ||
|
||
@misc{JuliaBUGS, | ||
title = {JuliaBUGS.jl}, | ||
url = {https://github.com/TuringLang/JuliaBUGS.jl} | ||
} |
15 changes: 15 additions & 0 deletions
15
news/posts/DoodleBUGS-Introduction/university-of-york-ieee.csl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
<?xml version="1.0" encoding="utf-8"?> | ||
<style xmlns="http://purl.org/net/xbiblio/csl" version="1.0" default-locale="en-GB"> | ||
<!-- This style was edited with the Visual CSL Editor (http://editor.citationstyles.org/visualEditor/) --> | ||
<info> | ||
<title>University of York - IEEE</title> | ||
<id>http://www.zotero.org/styles/university-of-york-ieee</id> | ||
<link href="http://www.zotero.org/styles/university-of-york-ieee" rel="self"/> | ||
<link href="http://www.zotero.org/styles/ieee" rel="independent-parent"/> | ||
<link href="https://subjectguides.york.ac.uk/referencing-style-guides/ieee" rel="documentation"/> | ||
<category citation-format="numeric"/> | ||
<category field="engineering"/> | ||
<updated>2025-07-11T09:55:45+00:00</updated> | ||
<rights license="http://creativecommons.org/licenses/by-sa/3.0/">This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License</rights> | ||
</info> | ||
</style> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -227,6 +227,22 @@ | |
# GSOC Contributors: | ||
|
||
- title: Google Summer of Code Students | ||
subtitle: GSoC 2025 | ||
members: | ||
- name: Shravan Goswami | ||
image: shravan-goswami.jpg | ||
university: Uka Tarsadia University | ||
gsoc_project: "DoodleBUGS: a Browser-Based Graphical Interface for JuliaBUGS" | ||
gsoc_project_link: | ||
gsoc_report: /news/posts/2025-08-28-DoodleBUGS-Introduction/index.html | ||
links: | ||
- website: https://shravangoswami.com/ | ||
- github: https://github.com/shravanngoswamii | ||
- twitter: https://twitter.com/shravangoswamii | ||
- linkedin: https://www.linkedin.com/in/shravangoswami/ | ||
- mail: [email protected] | ||
|
||
- title: | ||
shravanngoswamii marked this conversation as resolved.
Show resolved
Hide resolved
|
||
subtitle: GSoC 2023 | ||
members: | ||
- name: Zuheng(David) Xu | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.