R crashes randomly when computing posterior draws, forecasts and fevds

[DavidZenz]

Hey,

First off, thanks for your work on this great package - it's been super helpful!

I'm running into a strange issue where R crashes at seemingly random times when computing model outputs such as:

bsvars::compute_fitted_values(out)
bsvars::forecast(out, horizon = 8)
bsvars::compute_variance_decompositions(out, horizon = 8)

However, the following function runs without any issues every time:

bsvars::compute_impulse_responses(out, 8)

works all the time.

What's puzzling is that the crash doesn't happen consistently at the same point - sometimes later in my loop, sometimes sooner (often very soon). It appears to be completely random.

> R.version
               _                           
platform       x86_64-pc-linux-gnu         
arch           x86_64                      
os             linux-gnu                   
system         x86_64, linux-gnu           
status                                     
major          4                           
minor          3.0                         
year           2023                        
month          04                          
day            21                          
svn rev        84292                       
language       R                           
version.string R version 4.3.0 (2023-04-21)
nickname       Already Tomorrow  

> sessionInfo()
R version 4.3.0 (2023-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 10 (buster)

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3.10.3 
LAPACK: /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3.10.3;  LAPACK version 3.8.0

locale:
 [1] LC_CTYPE=de_AT.UTF-8       LC_NUMERIC=C               LC_TIME=de_AT.UTF-8        LC_COLLATE=de_AT.UTF-8     LC_MONETARY=de_AT.UTF-8    LC_MESSAGES=de_AT.UTF-8    LC_PAPER=de_AT.UTF-8      
 [8] LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=de_AT.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Vienna
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] sandwich_3.1-1     utf8_1.2.4         generics_0.1.3     GIGrvg_0.8         lattice_0.20-45    magrittr_2.0.3     countrycode_1.4.0  grid_4.3.0         pkgload_1.3.4     
[10] RcppProgress_0.4.2 nnet_7.3-18        bsvars_3.2         forecast_8.23.0    fansi_1.0.6        scales_1.3.0       cli_3.6.2          rlang_1.1.5        munsell_0.5.0     
[19] tools_4.3.0        parallel_4.3.0     coda_0.19-4.1      dplyr_1.1.4        colorspace_2.1-0   ggplot2_3.4.4      curl_6.2.2         vctrs_0.6.5        R6_2.5.1          
[28] strucchange_1.5-4  zoo_1.8-13         lifecycle_1.0.4    tseries_0.10-58    fs_1.6.3           MASS_7.3-58.3      pkgconfig_2.0.3    urca_1.3-4         stochvol_3.2.5    
[37] pillar_1.9.0       gtable_0.3.4       glue_1.7.0         vars_1.6-1         quantmod_0.4.25    Rcpp_1.0.14        RcppTN_0.2-2       tibble_3.2.1       lmtest_0.9-40     
[46] tidyselect_1.2.0   rstudioapi_0.15.0  nlme_3.1-162       xts_0.13.2         timeDate_4041.110  fracdiff_1.5-3     compiler_4.3.0     quadprog_1.5-8     TTR_0.24.4 

If helpful, I can try to provide a reproducible example - just let me know!

Thanks again for your support and all the work you're putting into bsvars and bsvarSIGNs.

[adamwang15]

Hi @DavidZenz

Thanks for your feedback! We have not experienced R crashing with those 3 functions.

Yes, please provide a reproducible example, that would be very helpful!

[donotdespair]

Hi David @DavidZenz

Thanks for reaching out. Would you please provide the whole script and the data files, so we can reproduce the error you get? Without this, we can only guess. The closer your script is to the standards of Minimal reproducible example the more informative it is for us. Thanks.

Greetings, Tomasz @donotdespair

[DavidZenz]

Hey both,

here a minimal reproducible example, which crashes randomly during the loop

rm(list = ls())
invisible(gc())

library(bsvars)
library(bsvarSIGNs)

# Load ts_data directly
ts_data <- readRDS("minimal_reproducible_example.rds")

# Sign restriction matrix
n_vars <- ncol(ts_data)
sign_irf <- matrix(NA, nrow = n_vars, ncol = n_vars)
rownames(sign_irf) <- colnames(ts_data)
colnames(sign_irf) <- paste0("shock_", colnames(ts_data))

for (i in 2:n_vars) {
  short_code <- sub("^d_hr_", "", colnames(ts_data)[i])
  if (short_code %in% c("fxlcu", "wag")) {
    sign_irf[i, 1] <- NA
  } else {
    sign_irf[i, 1] <- 1
  }
}

for(lag in 1:8) {
  
  message(paste0("Lag: ", lag))
  
  # Estimate and plot
  spec <- specify_bsvarSIGN$new(ts_data, p = lag, sign_irf = sign_irf)
  out <- estimate(spec, 50000)
  
  fitted <- bsvars::compute_fitted_values(out)
  plot(fitted)
  
  forecasted <- bsvars::forecast(out, horizon = 8)
  plot(forecasted)
  
  fevd <- bsvars::compute_variance_decompositions(out, horizon = 8)
  plot(fevd)
}

find the corresponding data here: https://github.com/DavidZenz/bsvarSIGNs-example/blob/main/minimal_reproducible_example.rds

[DavidZenz]

Dear @donotdespair,
could you reproduce the crashes?

[donotdespair]

Hi David @DavidZenz

No, I haven't really. We postponed any significant developments on bsvarSIGNs to June-July. This is when we might attend it. We are both too busy to with other projects at the uni.

But we are coming back to this.

Greetings, Tomasz @donotdespair

[georgetyler]

To add a +1 to this: I've experienced "SEGFAULT: 'memory not mapped'" errors immediately after running compute_historical_decompositions() and compute_fitted_values() (and similar to @DavidZenz, never after compute_impulse_responses(), which runs immediately).

In my experience it appears to be related in some way to the size of the posterior object created by estimate(), as after I manually and aggressively thin this object these commands often work. (For reference the model I am estimating is 5 variables, 21 restrictions, ~ 7000 observations over 1000 draws; thinning down to 200 draws often avoids a segfault). These segfaults occur even though the displayed size of the estimate() environment is not overly large - about 250MB - and my computer has plenty of available RAM. I have replicated it over separate machines.

My apologies for not providing a reprex, but I hope this anecdotal evidence is still slightly useful.