Add gmrf and precision builder functions as precursors for eventually adding DSEM by e-perl-NOAA · Pull Request #1594 · NOAA-FIMS/FIMS

e-perl-NOAA · 2026-06-30T14:29:51Z

Disclaimer
I don't actually know what I'm doing here, but there was enthusiasm for progress being made on getting DSEM into FIMS so I embarked on this journey ⛵ with nothing but Copilot and a dream ☁️. Please keep that in mind as you make comments and I beg you to explain things in a way that you would see them in those "XXX for Dummies" books.

What is the feature?

Getting the behind the scenes functions of gmrf and a precision builder (that can be used with more than just dsem) that are precursors seemed like a good point to start merging things in since myself and copilot/notebookLM then all started to get confused on how to proceed from here (it started going a bit off the rails once we started getting to what needed to be added in the information.hpp file) . I don't even think that these functions are "exposed" enough to write tests for? Maybe I'm wrong here and if so, @Bai-Li-NOAA I would DEFINITELY need your help.

Once it seems like there is consensus that these are a good way forward and/or good to go, they can be merged in and I/we can incrementally work on getting DSEM into FIMS in subsequent PRs. Hopefully that method confuses copilot/notebookLM (and honestly, myself) less.

How have you implemented the solution?

Add gmrf function
Add precision builders function - this can eventually take more than just DSEM

Instructions for code reviewer

👋Hello reviewer👋, thank you for taking the time to review this PR!

Please use this checklist during your review, checking off items that you have verified are complete but feel free to skip over items that are not relevant!
See the GitHub documentation for how to comment on a PR to indicate where you have questions or changes are needed before approving the PR.
Please use standard conventional messages for both commit messages and comments
PR reviews are a great way to learn so feel free to share your tips and tricks. However, when suggesting changes to the PR that are optional please include nit: (for nitpicking) as the comment type. For example, nit: I prefer using a data.frame() instead of a matrix because ...
Engage with the developer. Make it clear when the PR is approved by selecting the approved status, and potentially commenting on the PR with something like This PR is now ready to be merged.

Checklist

The code is well-designed
The code is designed well for both users and developers
Code coverage remains high- [ ] Comments are clear, useful, and explain why instead of what
Code is appropriately documented (doxygen and roxygen)

This file implements the GMRF distribution functor, which evaluates the log-likelihood of Gaussian Markov Random Fields using a sparse precision matrix.

github-actions · 2026-06-30T14:30:06Z

🎨 Chore: code formatting workflow

Our automated workflows cannot run on forks because of permission issues, and thus, we ask that you run the following code locally and push any changes that are created to your feature branch. You will only be reminded of this once per PR. Thank you!

Format C++ code

Install clang-format version 18.0.0

Run the following command from the repository root:

clang-format -i --style="{BasedOnStyle: Google, SortIncludes: false}" $(find ./inst/include ./src ./tests/gtest -name "*.hpp" -o -name "*.cpp")

Format R code

Install {styler} and {roxygen2}
Run the following commands in R from the repository root:

styler::style_pkg() # Style R code
roxygen2::roxygenise() # Update documentation
styler::style_pkg() # Style R code again
roxygen2::roxygenise() # Update documentation again
usethis::use_tidy_description() # Style DESCRIPTION file

Push changes

Commit the formatting with a commit message of "Chore: format feature branch"
Push to your fork

Bai-Li-NOAA · 2026-06-30T15:33:09Z

I don't even think that these functions are "exposed" enough to write tests for? Maybe I'm wrong here and if so, @Bai-Li-NOAA I would DEFINITELY need your help.

Wow 😮 , you did it! I haven't taken a close look at the code in those files yet, but for gmrf.hpp and precision_builder.hpp, you could write C++ unit tests using FIMS:::use_gtest_template(). If you'd like, here are some additional instructions for Fernando.

For the rcpp_*.hpp files, yes, they'll need to be added to inst/include/interface/rcpp/rcpp_interface.hpp and exposed to R. After that, we can write testthat tests for those modules. You can find an example here.

I'm not sure what your timeline is for working on the tests, but I'll be setting aside dedicated time each week to work on testing-related issues. Feel free to join those sessions (starting July 9 and listed on the FIMS calendar) if you have any questions or would like to work through any testing challenges together!

e-perl-NOAA · 2026-06-30T15:57:02Z

@Bai-Li-NOAA Yeah, that sounds great! I won't have time to work on tests this week, but I'll definitely have questions for you when I start working on it next week! Also, I love that you are leaning into me naming my AI/agents Fernando 🤣

kellijohnson-NOAA · 2026-07-14T17:31:03Z

@nathanvaughan-NOAA can you review this PR?

e-perl-NOAA · 2026-07-14T17:43:09Z

@kellijohnson-NOAA I need to add tests, I just haven't had time to work on them 😞

nathanvaughan-NOAA · 2026-07-14T18:20:59Z

No worries @e-perl-NOAA, Fernando is a busy guy I'm sure he'll get to them eventually 😁 I'll just go through the functions.

Andrea-Havron-NOAA · 2026-07-14T20:07:49Z

+    /**
+     * @brief Evaluates the GMRF log probability density function.
+     * @details The evaluation follows: 
+     * \f[ \log(f(x)) = -\frac{1}{2} (x - \mu)^T Q (x - \mu) + \frac{1}{2} \log|Q| \f]


The formula is missing the constant term: -N\log(\sqrt(2\pi))
N is the size of the latent state.

Andrea-Havron-NOAA · 2026-07-14T22:00:41Z

@e-perl-NOAA, there are two issues:

beta_z is a parameter vector that is used to both assemble and scale the precision matrix. We can simplify code by keeping beta_z a member of the dsem module ONLY and use the value to scale the precision matrix in the GMRF call.
There isn't any code linking the DSEMPrecisinMatrixBuilder to Q in the GMRF.

I started working on a comment that included a bunch of code - I realized it might be easier to set up a co-working session to go over these changes together. I think Fernando has gotten you to a nice spot, but at this point, writing more instructions to AI would probably take more time than implementing the needed changes directly!

Andrea-Havron-NOAA · 2026-07-14T22:02:21Z

+        }
+
+        // Register builder in Information Map (to be accessed by GMRF distribution)
+        info->dsem_builders[this->id] = builder; 


dsem_builders does not exist in information. More code is needed to link the precision matrix builder to the GMRF both here and in information.

Andrea-Havron-NOAA · 2026-07-14T22:03:23Z

 };

+/**
+ * @brief Rcpp interface for DSEM growth.


remove beta_z from the next four files (rcpp_growth, rcpp_maturity, rcpp_recruitment, and rcpp_selectivity)

Andrea-Havron-NOAA · 2026-07-14T22:04:26Z

@@ -0,0 +1,75 @@
+/**


this file can be removed, beta_z will be linked in a different way

nathanvaughan-NOAA · 2026-07-16T15:45:19Z

+        }
+
+        // Centering: x - mu because TMB's GMRF expects input centered around a mean of 0.
+        vector<Type> x_centered(n_x);


I think this needs to be std::vector

It would also be faster to have

vector x_centered;
x_centered.reserve(n_x);
for (size_t i = 0; i < n_x; ++i) {
x_centered.emplace_back(this->get_observed(i) - this->get_expected(i));
}

this avoids filling all the x_centered values with default first and overwriting them, emplace also avoids creating and then moving the centered value which push_back does.

nathanvaughan-NOAA · 2026-07-16T17:51:19Z


+/**
+ * @brief Rcpp interface for DSEM growth.
+ */


We should avoid setting this up to create module specific DSEM classes. Everything should be handled in the dsem interface so it can then point to arbitrary variables in any current/future modules without adding development overhead to them.

nathanvaughan-NOAA · 2026-07-16T17:55:26Z

+        int type = 0;         /**< 1 = A path effect (Rho), 2 = Variance (Gamma) */
+        int from = 0;         /**< The variable the arrow starts from */
+        int to = 0;           /**< The variable the arrow points to */
+        int beta_index = 0;   /**< Which parameter in beta_z to use for this arrow */


Am I right that using the index here is basically so you can map off multiple linkages to be estimated together? I'm not sure how common that would be but it may make more sense just to have the value in RAMPath and allow it to be estimable and/or mapped through TMB like other parameters? That basically replaces beta_index and start with beta_z that can be a fixed or estimated variable.

nathanvaughan-NOAA · 2026-07-16T18:09:30Z

+            }
+
+            // Poke the "strength" into the correct grid slot based on arrow type.
+            if (this->paths[r].type == 1) {


When I looked into SparseMatrix it looks like using .coeffRef is supposedly really slow due to memory moving issues and using Eigen::Triplet is much faster. Something like this apparently

int rows = 3; int cols = 3; Eigen::SparseMatrix<double> A(rows, cols); std::vector<Eigen::Triplet<double>> tripletList; tripletList.reserve(4); tripletList.push_back(Eigen::Triplet<double>(0, 0, 1.5)); tripletList.push_back(Eigen::Triplet<double>(1, 2, 2.0)); tripletList.push_back(Eigen::Triplet<double>(2, 1, -3.5)); tripletList.push_back(Eigen::Triplet<double>(2, 2, 4.0)); A.setFromTriplets(tripletList.begin(), tripletList.end()); A.makeCompressed();

nathanvaughan-NOAA · 2026-07-16T19:23:41Z

+        }
+
+        // Centering: x - mu because TMB's GMRF expects input centered around a mean of 0.
+        vector<Type> x_centered(n_x);


It would also be faster to have

vector x_centered;
x_centered.reserve(n_x);
for (size_t i = 0; i < n_x; ++i) {
x_centered.emplace_back(this->get_observed(i) - this->get_expected(i));
}

this avoids filling all the x_centered values with default first and overwriting them, emplace also avoids creating and then moving the centered value which push_back does.

e-perl-NOAA and others added 11 commits June 30, 2026 10:10

Add GMRF distribution functor implementation

b2dd9d7

This file implements the GMRF distribution functor, which evaluates the log-likelihood of Gaussian Markov Random Fields using a sparse precision matrix.

fix gmrf

f8f8739

update wordlist with gmrf

e4db7ce

add precision builder

ab4fcf7

add GMRF and Eigen back in

64da0d8

some additional notes

9ef8ede

add beta_z to Fims object; add modular specific functors for dsem

f5ee876

all changes pushed until I can get back to working on this

5a897c1

make changes thus far

aef341e

return information.hpp to previous way

0ed8613

add eigen back in

47d2374

e-perl-NOAA requested review from Andrea-Havron-NOAA, Bai-Li-NOAA, grantdadams and kellijohnson-NOAA June 30, 2026 14:29

fix wordlist for spell check action

c9c1b3b

kellijohnson-NOAA requested review from nathanvaughan-NOAA and removed request for Andrea-Havron-NOAA, Bai-Li-NOAA and grantdadams July 14, 2026 17:30

Andrea-Havron-NOAA reviewed Jul 14, 2026

View reviewed changes

nathanvaughan-NOAA reviewed Jul 16, 2026

View reviewed changes

kellijohnson-NOAA force-pushed the main branch from eb122ca to fd8901b Compare July 18, 2026 19:54

Uh oh!

Conversation

e-perl-NOAA commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the feature?

How have you implemented the solution?

Instructions for code reviewer

Checklist

Uh oh!

github-actions Bot commented Jun 30, 2026

🎨 Chore: code formatting workflow

Format C++ code

Format R code

Push changes

Uh oh!

Bai-Li-NOAA commented Jun 30, 2026

Uh oh!

e-perl-NOAA commented Jun 30, 2026

Uh oh!

kellijohnson-NOAA commented Jul 14, 2026

Uh oh!

e-perl-NOAA commented Jul 14, 2026

Uh oh!

nathanvaughan-NOAA commented Jul 14, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Andrea-Havron-NOAA commented Jul 14, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

e-perl-NOAA commented Jun 30, 2026 •

edited

Loading