Repository navigation
Progress bar #18
Description
CmdStanPy has a nice progress bar that we can implement in R too.
We are currently using processx::run(..., echo=TRUE) so that the R user sees the output directly from CmdStan, but processx provides a lot more control than that and it looks like it has what we’d need behind the scenes for a progress bar even for chains in parallel.
Activity
jgabry commentedon Oct 25, 2019
After discussing with @mitzimorris I found out that maybe there are some problems with the progress bar implementation inn CmdStanPy so maybe we shouldn't try to copy that until it's sorted out there.
Also, an alternative to a progress bar would be to run background processes and provide a way for the user to check the status. A progress bar isn't necessary for models that run fast, and maybe for slow models it's better to let them just run in the background and allow the user to check the status when they want? This is possible with the processx package we're already using.
Anyway, I haven't thought too hard about this yet so I'm definitely open to other ideas!
rok-cesnovar commentedon Nov 11, 2019
So a progress bar would be fairly simple to implement after #61 is merged, provided that a user doesnt specify refresh=0. @jgabry do we still want to do this?
jgabry commentedon Nov 11, 2019
I could go either way on this. The current output from parallelization is fine, although a progress bar could be nicer. What do you think?
rok-cesnovar commentedon Nov 11, 2019
Do you have an example for an R package that uses progress bars? I am not completely sure how it would look. Worth a try tho.
jgabry commentedon Nov 11, 2019
Yeah it's worth a try. Things to check out whenever you want to dive into this (definitely not super urgent though):
progresspackage (https://github.com/r-lib/progress, https://cran.r-project.org/package=progress). Has some of same very capable authors as theprocessxpackage that we're using.revdepcheckpackage (https://github.com/r-lib/revdepcheck, for testing reverse dependencies) and that has a really nice progress bar. Looks like it uses theprogresspackage.progressby looking at it's CRAN page: https://cran.r-project.org/package=progress (scroll down to reverse dependencies section). There are a bunch so you may be able to look at those packages for implementation details.There are other packages for making progress bars but I haven't used them. I've heard it can be difficult to do progress bars for parallelized code in R, but I haven't gone down that rabbit hole so I don't know any details about what the challenges are unfortunately.
mike-lawrence commentedon Aug 1, 2020
What do folks think of the approach taken by my package ezStan where I simply watch the contents of the CSV files to establish progress (inc. reporting post-warmup divergences and permitting termination on detection of these)?
rok-cesnovar commentedon Aug 2, 2020
@mike-lawrence that sounds like a great idea.
So you essentially count the number of lines in the CSV? Do you read the entire CSV in a separate thread constantly for checking divergences?
mike-lawrence commentedon Aug 2, 2020
Precisely. My package is a bit of a monstrosity of poorly-commented/hack-y code, so if this kind of progress indicator is of interest for cmdstanr I'd probably start from scratch. But it would be neat to finally contribute to the project beyond my forum activity, so I'm keen to do it.
rok-cesnovar commentedon Aug 2, 2020
Cool! I am definitely in favor.
Another option that would only allow showing progress (not divergences and other diagnostic realtime stuff) is to rely on stdout. If the progress bar is enabled then set refresh = 1. That means that we get information for each iteration.
If a user wants realtime reports for divergence/treedepth then we need to read the CSV.
mike-lawrence commentedon Aug 2, 2020
Good idea, not monitoring the csv if the user hasn't asked for any csv-content-dependent behaviour will make things more resource-friendly during sampling. I'll have to teach myself how to monitor the stdout instead, but I suspect that's within my capacity.
rok-cesnovar commentedon Aug 2, 2020
For sampling we have a process function with a "state machine" with which we don't do much, mostly just limit that the initial Cmdstan metadata dump is not printed and to mark the transition from warmup to sampling and such: https://github.com/stan-dev/cmdstanr/blob/master/R/run.R#L650
This is called every 200ms I believe if any stdout came in.
For all the other methods the function is here https://github.com/stan-dev/cmdstanr/blob/master/R/run.R#L791
Here we only make sure the Cmdstan metadata dump is not printed.
jgabry commentedon Aug 2, 2020
Sounds great!
mike-lawrence commentedon Aug 18, 2020
Just a heads up that I'll start working on this today. I'd somehow gotten it into my head that I was waiting on some info but see that all I need is in Rok's last comment. I'll start with the simple progress bar that watches stdout, then once that's working solidly dive into the csv-informed monitor.
Tangentially, now that I've seen that the ArViz crew is using HDF5/NetCDF, I'm more confident that that's at least not a completely ridiculous thing to work on as a replacement for csv in cmdstan, so I may look at that in parallel to the monitor (which would benefit something that's not csv).
23 remaining items
jgabry commentedon Dec 5, 2025
Sounds good, thanks!
josswright commentedon Dec 7, 2025
Proof of concept.
I played with a few ideas for how to handle basic reporting. At the moment, I've added two arguments to
$sample():show_progress_bar: Default =FALSE.If
TRUE, registers a progress bar viaprogressrand signals an update for every output line that matches"Iteration:". The user is responsible for registering those progress updates with an appropriate handler. (In the video below I'm usingcli's progress bar, but there are quite a few alternatives, including RStudio's jobs pane or even audio updates viabeepr.)suppress_iteration_messages: Defaults to the value ofshow_progress_bar, but can also be set directly.If
TRUE, disables display of output lines matching"Iteration:", while still causing the progress bar to update. This keeps all of Stan's other informative output, but just removes the superfluous iteration messages when using a progress bar. Felt like a clean default.(I show both options in use in the video.)
One thing that might be worthwhile is to report the text of the line causing the update to the progress bar. That would give the end user the option of incorporating the text of the line into the progress bar, for example as a status message, if they feel like it. For the moment, though I wanted to keep things relatively simple. (It's trivial to add this.)
I've tried to keep all additions to the code as non-intrusive as possible. It's still more of a proof of concept at the moment--although it has been working fine for me. I'll still need to do some stress testing to check that I haven't accidentally broken something.
Happy to hear comments as to whether this seems like a reasonable approach.
stan_progress.mp4
jgabry commentedon Dec 8, 2025
Very cool! Thanks for working on this. Some comments:
This is great. In the doc for this argument we can also point users to the alternatives you mentioned here. I think it's nice to have all these different options even if it means they need to register it themselves.
Yeah that does seem like a clean default. In general I don't love when the default of one argument depends on another argument, but in this case I think it makes sense.
I'd be curious to see what that looks like. Seems like it could sometimes be useful.
Do you mind if I post on the Stan forum that you're implementing this and point to this issue where people can give you feedback? I think it would be good to get more people to comment on this design and then also eventually try it out and give feedback once it's ready but before it's merged. (Or you're more than welcome to make the post yourself of course.)
Thanks again for working on this!
josswright commentedon Dec 9, 2025
Thanks. :)
I suppose it might be possible to have a convenience
register_cmdstanr_progress()function to set up a default handler appropriately formatted for Stan.I think it's probably worth passing the information across, even if the user doesn't display it. I'll have a play.
Sure. Happy to see what people think, so please feel free. I'm a little snowed under at the moment, but I'll tidy things up and push it to my fork as soon as I can.
Selfish reasons! :)
jgabry commentedon Dec 9, 2025
I'm in no rush, so we could wait to ask for feedback until you have something you want people to try out. Or do you prefer feedback on these posts describing what you're working on? Either way is fine by me.
josswright commentedon Dec 22, 2025
I've put a fork up here, under a
progressrbranch: https://github.com/josswright/cmdstanr/tree/progressrSmall updates:
CmdStanProcsso that the progress bar can update 'in line' with the iterations. (Instead of having a total of(iter_warmup+iter_sampling)/refresh)steps in the progress bar, with each tick updating by one, it now has(iter_warmup+iter_sampling)steps, and each tick of the progress bar updates it byrefresh. It required an extra parameter passed through, but felt neater.stdoutis now sent to the progress bar as a 'message', regardless of whether the progress step needs to be updated. This can be displayed in the progress bar if the end user chooses to do that. (There isn't really a good way to do that right now that I can see with the existing popular progress bar packages--you can include it incliwith the{cli::pb_status}parameter, but it makes for very wide lines--but it's there for anyone who wants it.)register_default_progress_handler()function intoutils.Rthat sets up a default progress bar and registers it as a global handler forprogressrupdates from the sampling process. Just avoids a chunk of boilerplate code for users who want to display a progress bar without complications.Haven't had a chance to stress test this much yet, but happy for people to poke at it.
josswright commentedon Dec 22, 2025
For my own reference, a couple of things that occurred to me:
deregister_default_progress_handler()function.VisruthSK commentedon Jan 13, 2026
@josswright Hi! I took a look at your fork and it looks good! I have a few small things I'd want to change, but it looks great on the whole. Do you want to open a PR?
jgabry commentedon Jan 13, 2026
Sorry for not replying sooner, it's been a hectic couple of weeks. Thanks for putting up the fork people can install from. I had wanted to get more people to try it out but I haven't posted on the Stan forum yet. If we post on the forum @josswright do you have a few basic instructions for people to try it out so they don't have to comb through our back and forth here in this issue to figure it out? Thanks again for working on this!
josswright commentedon Jan 13, 2026
Hi. No problem! Life has been incredibly hectic here as well, and still will be for at least a week or so.
Happy to have it posted to the Stan forum. I'll pull out a description of basic usage some time in the next day or so. (I think I wrote it up in one of my commits to the repository, so it's probably more or less there already.)
I have run into an occasional bug where (I think) an error occurs at the end of a sampling run due to a progress update happening after the bar has been closed. I hadn't quite worked out exactly what causes it, so was holding off on making a pull request until I had a chance to dive into it.
jgabry commentedon Jan 13, 2026
Sounds good! No rush on the PR, whenever you think you're ready.
josswright commentedon Feb 13, 2026
Hi all. Apologies for the long delay in updating. A fair amount of life and term happened all of a sudden. I've now submitted a pull request for discussion.
Getting the progress bar to update neatly in line with CmdStan's messages was surprisingly fiddly in the end, as there are some odd edge cases in when iterations are printed. I'm fairly confident that this is now working robustly. I did have to pass
iter_warmupthrough to theCmdStanProcsobject, and passediter_samplingthrough while I was at it, but that didn't seem too terrible a change.I've spent the evening testing various different and odd values for
refreshto stress test whether the progress bar updates correctly in every situation I could think to throw at it. I haven't written any formal unit tests, though, so there's probably some more checking to be done.I've also defaulted to the progress bar closing automatically when it reaches its predetermined number of steps, as I really don't want the progress bar to crash at the end of someone's multi-day sampling run and take their model with it.
Very happy to have this pulled apart.
I initially toyed with the idea of setting
refreshto1by default if the progress bar was enabled and normal iteration messages tostdoutwere suppressed, but in the end it felt too much of an invasive change to CmdStan's default options and not worth the complication. (As far as I can see from my own testing, though, manually settingrefreshto1for a smoothly-updating progress bar doesn't cause any detectable slowdown.)jgabry commentedon Oct 12, 2026
Experimental progress bar merged (in #1138)! Currently available on the v1.0 branch.