Skip to content

Add Friedland Chapter 7 Jupyter notebook - #1189

Open
salexanian wants to merge 4 commits into
casact:mainfrom
salexanian:friedland-ch7-notebook
Open

Add Friedland Chapter 7 Jupyter notebook#1189
salexanian wants to merge 4 commits into
casact:mainfrom
salexanian:friedland-ch7-notebook

Conversation

@salexanian

@salexanian salexanian commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary of Changes

This PR replaces the Chapter 7 .rst documentation with a Jupyter notebook.

Current progress:

-Migrated Exhibits I and II.
-Added Exhibit III.
-Updated the supporting data.

Remaining work includes:

-Exhibit IV.
-Assertions against Friedland.
-Formatting and notebook cleanup.

Related GitHub Issue(s)

#588

Additional Context for Reviewers

I will post a follow-up comment with several implementation observations and questions.

Checklist

  • I passed tests locally for both code (uv run pytest) and documentation changes (uv run --directory docs jb build . --builder=custom --custom-builder=doctest)

Note

Low Risk
Documentation navigation only; no runtime code or library API changes in this diff.

Overview
Updates the Jupyter Book table of contents so Friedland Chapter 7 is published as two sibling sections instead of a single entry.

Chapter 7 - Part I keeps friedland/chapter_7.rst and adds an explicit TOC title. Chapter 7 - Part II is a new section pointing at friedland/chapter_7_part_2.ipynb, which wires the second notebook into the Friedland chapter navigation alongside the existing Chapter 6, 9, and 10 materials.

Reviewed by Cursor Bugbot for commit 9be61ee. Bugbot is set up for automated code reviews on this repo. Configure here.

@salexanian

Copy link
Copy Markdown
Contributor Author

I have been away from this issue for some time so please let me know if what I am doing is now obsolete/no longer required. If it is not, I hope to continue to help you and @priyam0k put this issue to rest.
[friedland_ch7_USPP.xlsx](https://github.com/user-

@salexanian your workbook isn't wasted. i think the useful role for it is as the source for the hardcoded numbers in the asserts, especially where friedland's printed figures are rounded. the reconciliation still has to happen in the notebook, but the numbers have to come from somewhere trustworthy.
i went through the same USPP scenarios for chapter 10's exhibit III, so a few things that saved me time:

  • use IPython.display.display() on the triangle or dataframe instead of print
  • display the rounded values like the text, but project ultimates off the unrounded estimator, otherwise steady state stops reconciling exactly
  • one cell at the bottom with np.isclose / np.allclose per exhibit. i've been using rtol 5e-3 for totals and atol 1e-3 for cdfs and ratios

one thing i'd value your read on. steady_state and increasing_claim reconcile to the text for me, but increasing_case and increasing_claim_case don't. i think those two CSVs need corrections. if your workbook reproduces the text for them, that settles it and i'll raise the data fix.
also, so we don't write the same cells twice: i have exhibit III and IV drafted in the old rst form. since the direction is notebooks now, i can hand those over as raw material or port them myself, whichever you prefer. your call.

Hi @priyam0k - thank you for the helpful advice! I will check my code and ensure it follows what you listed above - I will send you and @henrydingliu my notebook to ensure that I am conforming with your approach. Re: the notebook - at least I am getting some Exam 5 practice out of it!

I will re-check my results against Friedland, and will look at your reconciliation as well for Exhibit III - I will add the asserts to my notebook and send you both the results. I recall there being some small discrepancies as well, but I will give you the details shortly. I would also be glad to use your code for Exhibit IV if you have it.

Hi @henrydingliu and @priyam0k , I attach my work-in-progress notebook for Ch. 7. It currently contains the following:

  1. Exhibits I and II as prepared by @henrydingliu in his .rst, adapted to the Jupyter notebook.
  2. Exhibit III from my existing notebook.
  3. An empty section for Exhibit IV, which I will add using @priyam0k 's Chapter 10 code.

What remains to be done (which I am continuing to work on) is the following:

  1. Implement asserts comparing the generated results to those in Friedland. I will follow @priyam0k 's error tolerance conventions for these.
  2. Fix inconsistencies in the rendering of headings, subheadings, etc. across the Exhibits.
  3. Fix the formatting of the numbers/significant digits in the Exhibit III output (from my notebook).
  4. Add Exhibit IV.
  5. Simplify my Exhibit III code to conform to @henrydingliu and @priyam0k presentation style in their notebook sections.

I have the following questions/observations to make based on my work so far:

  1. There is a warning "UserWarning: Some exclusions have been ignored. At least 1 (use preserve = ...) link ratio(s) is required for development estimation." which I have been able to trace to the _drop_n and _drop_x functions in the DevelopmentBase class in base.py. Essentially, the warning is intended to remind a user to set the preserve parameter to ensure that the minimum intended number of ldf values are retained after dropping extreme values. Unfortunately, it appears that this warning flag will appear when a user sets preserve=1, which I think is a bug traceable the if statement in this code:
            if preserve == 1:
                warning = (
                    "Some exclusions have been ignored. At least "
                    + str(preserve)
                    + " (use preserve = ...)"
                    + " link ratio(s) is required for development estimation."
                )
            else:
                warning = (
                    "Some exclusions have been ignored. At least "
                    + str(preserve)
                    + " link ratio(s) is required for development estimation."
                )
            warnings.warn(warning)

If you agree that this is a bug or that the issue should be looked at further, I would be happy to take the appropriate steps. I had to supress this warning for the notebook to properly display some of the code without the warning.

  1. Where we define functions that generate tables and perform all the associated formatting, should this part of the code be visible in the docs, or should these cells be hidden? If they are to be shown, should we be developing a standard way of showing everything, formatting tables, etc.? I will look into this issue myself as I complete the notebook, but wanted to flag this.

  2. @henrydingliu had a helpful section in Exhibit I showing the unrounded and rounded ldfs. Unfortunately, when I moved the code to Jupyter from the .rst, the unrounded output now automatically rounds due to the following code in display.py, which reads as follows:

@staticmethod
    def _get_format_str(data: DataFrame) -> str:
        """
        Returns a numerical format string based on the magnitude of the mean absolute value of the values in the
        supplied DataFrame.

        Returns
        -------
        str
        """
        if np.all(np.isnan(data)):
            return ""
        elif np.nanmean(abs(data)) < 10:
            return "{0:,.4f}"
        elif np.nanmean(abs(data)) < 1000:
            return "{0:,.2f}"
        else:
            return "{:,.0f}"

Essentially, the default setting in Chainladder is to round HTML-rendered output to a prespecified number of decimal places for presentational quality depending on the average magnitude of the numbers (so based on the above the ldfs automatically round to 2 decimal places when rendered as HTML (which is what I think the notebook approach is doing). There are likely good reasons for the default display format above, but it does interfere with rendering of unrounded values in some circumstances (such as in the Jupyter notebook) and required a manual override. I raising this in case the default display code above needs to be reassessed.

  1. I am trying to add some explanation of the sheets in Exhibit III (their purpose, context etc.), but if this is not required/needed (you can see some of the wording I put into Exhibit III so far), I can remove it or simplify it.

  2. If I have done anything incorrectly in the notebook or should be following a specific protocol, please let me know and I will adjust accordingly.

I will send an update in the next couple of days, but in the meantime the work continues.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 15.0% of exported symbols fully typed (196 / 1305)

Known Ambiguous Unknown Total
Project (head) 196 110 999 1305

Other symbols referenced but not exported by chainladder: 13

Known Ambiguous Unknown Total
Other (head) 3 1 9 13

Symbols without documentation:

  • Functions without docstring: 315
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): no exported symbol type-completeness changes detected.

Comment thread docs/friedland/chapter_7.ipynb Outdated
Comment thread docs/friedland/chapter_7_part_2.ipynb Outdated
@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.14%. Comparing base (8461cfb) to head (9be61ee).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1189   +/-   ##
=======================================
  Coverage   91.14%   91.14%           
=======================================
  Files          91       91           
  Lines        5365     5365           
  Branches      681      681           
=======================================
  Hits         4890     4890           
  Misses        340      340           
  Partials      135      135           
Flag Coverage Δ
unittests 91.14% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@henrydingliu

Copy link
Copy Markdown
Member

one general comment first. if you are still working through things, keep the PR as a draft

There is a warning "UserWarning: Some exclusions have been ignored. At least 1 (use preserve = ...) link ratio(s) is required for development estimation."

it's an annoyance for sure. but we also value the transparency. just leave it there for the time being

Where we define functions that generate tables and perform all the associated formatting, should this part of the code be visible in the docs, or should these cells be hidden?

keep the code visible. as for standardization, i think having a bit of variety actually gives our repo more character, i.e. we have a vibrant group of collaborators with different coding styles. there's some stuff we can standardize after we are through all the chapters. but not a priority for the time being.

had a helpful section in Exhibit I showing the unrounded and rounded ldfs. Unfortunately, when I moved the code to Jupyter from the .rst, the unrounded output now automatically rounds due to the following code in display.py, which reads as follows:

just leave the rst for now. you can change the rst title to part I and have both parts on the toc

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit e809abb. Configure here.

Comment thread docs/_toc.yml Outdated
@priyam0k

priyam0k commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

great progress on this @salexanian! since you're actively working through exhibit iv and the assertions, feel free to convert this to a draft PR so you don't feel rushed by review notifications. tag us whenever it's ready for final review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants