Source code for uslib.products.immediate_annuity.SPIA_US_SEC.Data

# modelx: pseudo-python
# This file is part of a modelx model.
# It can be imported as a Python module, but functions defined herein
# are model formulas and may not be executable as standard Python.

"""Input data shared by every by-contract projection.

The four input CSVs are read here, **once per model**, and referenced from
:mod:`~.SPIA_US_SEC.Projection` as ``data``. :mod:`~.SPIA_US_SEC.Projection` is
parameterized by ``idx``, the position of a model point in the table, so each
``Projection[i]`` is a separate ItemSpace with its own cells cache; if the readers lived
there, every model point would re-read every file.
Holding them in an unparameterized Space reads each file once no matter how many
contracts are projected.

Inputs are **external files**: plain CSVs in the model folder's parent directory,
``products/immediate_annuity/``, rather than data stored inside the model. The model
folder therefore holds nothing but formulas — no ``_data/``, no IOSpec, no embedded
values — so a diff of the model shows logic changes only. This follows
``annuallife.TradLife_A``; contrast ``basiclife.BasicTerm_S``, which keeps its inputs
*inside* the model through modelx's IOSpec machinery.

The consequence worth knowing: **the model is not portable on its own.** Copying the
``SPIA_US_SEC`` folder without its parent's CSVs produces a model that reads and then
fails on first evaluation.

**Three of the tables are SPIA_US_S's.** The mortality table, the improvement scale and
the surrender-charge scale are the files :mod:`~.SPIA_US_S` reads, read here in place: one
copy of each table serves both models, so an edit to one reaches both. The model point
table is this model's own, ``model_point_table_sec.csv``; it sits beside
``SPIA_US_S``'s ``model_point_table.csv``, which this model never opens.

:func:`input_dir` resolves the directory from ``_model.path.parent`` at run time, so
the model works wherever the repository is checked out. Each table has a filename
Reference and a reader Cells:

========================  ==============================  ==========================
Reference                 Cells                           File
========================  ==============================  ==========================
model_point_file          model_point_table()             model_point_table_sec.csv
mort_table_file           mort_table()                    mort_table.csv
improvement_scale_file    improvement_scale()             improvement_scale.csv
surr_charge_file          surr_charge_table()             surr_charge_table.csv
========================  ==============================  ==========================

**The model point table is checked as it is read.** Six of its columns are the
in-force interface: ``policy_count`` and ``duration_mth``, as in
``basiclife.BasicTerm_SE``, and the four columns of a contract's opening state, each
covered life's survival at the valuation date, the commutation fraction taken by then and
the 10-year CMT's change since the annuity date. :func:`model_point_table` refuses a
table that lacks any of them, repeats a ``point_id``, holds a count, a duration or a month
of death no contract can have, or holds an opening state no contract can be in. The checks
run in turn, and the first that fails raises, naming the missing column or every point
that fails it. It also refuses an annuity to commence more than ``future_horizon_mth``
months after the valuation date, 120 months as shipped: future new business is written
over a stated horizon, so that no frame runs longer than the stop rule from the annuity
date plus that horizon. Set to 0, the Reference refuses future new business altogether.
Whether a contract still owes a payment at the valuation date is a rule of the product
rather than of the table, and ``Projection.proj_len`` checks it.

**So is the surrender-charge scale.** :func:`surr_charge_scale` reads its policy years by
value, once, and refuses a year that is not a whole number, a gap, a repeat or a blank
rate, naming the file; ``Projection.surr_charge_rate`` takes the scale's first and last
years and its rates from there, never from the raw column.

The two mortality tables are keyed by ``(age, sex)`` and the surrender-charge table by
``policy_year``. They are separate files on purpose: the technical notes require the
best-estimate basis to be the **2012 IAM Basic** table projected with **Projection Scale
G2**, two distinct objects with distinct provenance, and neither may be embedded in the
model — the shipped tables are illustrative **[std]** stand-ins. To swap in a licensed
basis, replace ``mort_table.csv`` with a same-schema file, or point ``mort_table_file``
at a different name before the model computes anything. No formula changes. The
improvement scale is swappable the same way, which is how the notes' first sensitivity —
a 50% / 150% scaling of tabulated improvement — is run. Replacing a file changes
:mod:`~.SPIA_US_S` too, which reads the same one.

**The vectors.** The readers return what the files hold, and no
:mod:`~.SPIA_US_SEC.Projection` cell but ``model_point``, a reporting cell, reads them.
The others read the arrays below instead, one element at a time, a
model point's attribute being the element at its position, ``data.premium()[idx]``. That
is what lets the model compile to C, as ``basiclife.BasicTerm_SC`` does: each array is a
numpy vector or table of int64 or float64, which compiles to a typed memoryview, never one
of bools, objects or text, and each check of a table is an int. Each model point column a
projection reads is one vector of the column's own name, read element by element as
``SPIA_US_S`` reads the point's value, ``float()`` of an amount or a rate, ``int()`` of an
age, a month count or a year, ``bool()`` of a flag; and where a point's value can be one
its projection cannot use, a 0/1 vector beside it, ``<column>_ok``, says whether it is
usable. The text the projection branches on is decoded into 0/1 flags, each with a
``has_*`` flag that says the value is one ``SPIA_US_S`` names. The mortality table and the
improvement scale, keyed by age and sex, are tabled by schedule, one per sex, and age,
padded from age 0, each point carrying the schedule of each life's sex; the surrender
charges are a vector by policy year, padded from year 0:

===================================  =======  =================  ==========================
Cells                                dtype    Shape              From
===================================  =======  =================  ==========================
point_id()                           int64    [point]            the table's index
policy_count()                       float64  [point]            ``policy_count``
duration_mth(), duration_mth_ok()    int64    [point]            ``duration_mth``
premium()                            float64  [point]            ``premium``
premium_ok()                         int64    [point]            ``premium``, its flag
premium_tax_rate()                   float64  [point]            ``premium_tax_rate``
premium_tax_rate_ok()                int64    [point]            ``premium_tax_rate``, its
                                                                 flag
annual_income()                      float64  [point]            ``annual_income``
annual_income_ok()                   int64    [point]            ``annual_income``, its
                                                                 flag
cola_rate()                          float64  [point]            ``cola_rate``
cola_rate_ok()                       int64    [point]            ``cola_rate``, its flag
survivor_pct()                       float64  [point]            ``survivor_pct``
survivor_pct_ok()                    int64    [point]            ``survivor_pct``, its flag
rating_factor()                      float64  [point]            ``rating_factor``
rating_factor_ok()                   int64    [point]            ``rating_factor``, its
                                                                 flag
primary_age(), primary_age_ok()      int64    [point]            ``primary_age``
joint_age(), joint_age_ok()          int64    [point]            ``joint_age``
certain_months()                     int64    [point]            ``certain_months``
certain_months_ok()                  int64    [point]            ``certain_months``, its
                                                                 flag
frequency(), frequency_ok()          int64    [point]            ``frequency``
annuity_year(), annuity_year_ok()    int64    [point]            ``annuity_year``
is_joint()                           int64    [point]            ``joint``, 1 on True
commutation_enabled()                int64    [point]            the column, 1 on True
issue_state_excludes_withdrawal()    int64    [point]            the column, 1 on True
qualified()                          int64    [point]            the column, 1 on True
has_form()                           int64    [point]            ``form``, 1 on one of five
is_period_certain()                  int64    [point]            ``form``, 1 on
                                                                 life_certain or
                                                                 certain_only
is_certain_only()                    int64    [point]            ``form``, 1 on
                                                                 certain_only
is_cash_refund()                     int64    [point]            ``form``, 1 on cash_refund
is_installment_refund()              int64    [point]            ``form``, 1 on
                                                                 installment_refund
has_reduction_trigger()              int64    [point]            ``reduction_trigger``, 1
                                                                 on either or primary
is_trigger_either()                  int64    [point]            the same, 1 on either
has_timing()                         int64    [point]            ``timing``, 1 on arrears
                                                                 or advance
is_arrears()                         int64    [point]            the same, 1 on arrears
has_mort_basis()                     int64    [point]            ``mort_basis``, 1 on table
                                                                 or scenario
is_scenario()                        int64    [point]            the same, 1 on scenario
death_mth_1(), death_mth_1_ok()      int64    [point]            ``death_mth_1``, -1 where
                                                                 blank
death_mth_2(), death_mth_2_ok()      int64    [point]            ``death_mth_2``, -1 where
                                                                 blank
lives_if_init_1()                    float64  [point]            ``lives_if_init_1``
lives_if_init_2()                    float64  [point]            ``lives_if_init_2``
commute_frac_cum_init()              float64  [point]            the column
cmt10_shift_init()                   float64  [point]            the column
mort_sexes()                         int      scalar             *mort_table.csv*, checked
mort_table_array()                   float64  [schedule, age]    *mort_table.csv*
mort_age_first(), mort_age_last()    int64    [schedule]         *mort_table.csv*
mort_schedule_1()                    int64    [point]            ``primary_sex``, joined; 0
                                                                 where not
mort_schedule_2()                    int64    [point]            ``joint_sex``, joined; 0
                                                                 where not
improvement_sexes()                  int      scalar             *improvement_scale.csv*,
                                                                 checked
improvement_array()                  float64  [schedule, age]    *improvement_scale.csv*
improvement_age_first()              int64    [schedule]         *improvement_scale.csv*
improvement_age_last()               int64    [schedule]         *improvement_scale.csv*
improvement_schedule_1()             int64    [point]            ``primary_sex``, joined; 0
                                                                 where not
improvement_schedule_2()             int64    [point]            ``joint_sex``, joined; 0
                                                                 where not
surr_charge_rate_array()             float64  [policy_year]      :func:`surr_charge_scale`
surr_charge_year_first()             int      scalar             :func:`surr_charge_scale`
surr_charge_year_last()              int      scalar             :func:`surr_charge_scale`
===================================  =======  =================  ==========================

Each table is numbered by its own sexes, in sorted order, from schedule 1, so the two
tables need not hold the same sexes: :func:`mort_sexes` and :func:`improvement_sexes`, each
the table's one check, an int, give the count. Schedule 0 is a row of padding whose first
age is 1 and whose last is 0, so that no age lies between them: a life whose sex a table
lacks, a blank among them, has schedule 0 there, and ``Projection``'s reader of the table
refuses it, naming the point, the age and the life, at the first age it reads, as
``SPIA_US_S``'s ``.loc`` raised ``KeyError`` there. A point on the scenario basis, or a
single-life point's second life, never reads it. :func:`surr_charge_scale` stays the
scale's checked DataFrame, as at stage ``E``, and the vector and its two extents are read
from it.

**An array is read by element, never whole,** in this Space as in
:mod:`~.SPIA_US_SEC.Projection`. The compiler types an array cell as a memoryview only once
it has seen that every use of it, in either Space, is an element read: one ``len()``,
slice, loop over it or numpy call on it anywhere, and the cell compiles as a Python
object. So an array derived here reads the table's column, and a width comes from a table
or a check's int, never from another cell's array. Each age table's check is one int cell
that its array, its extents and its schedules read first, so that a projection, which
reads an extent first, meets the same refusal as a reader of the array. The column
keywords of :func:`model_point_table`'s ``DataFrame.assign`` are passed as ``**{...}``,
since a keyword named as a cell of this Space, ``death_mth_1``, would be rewritten by the
export.

**What a point's projection cannot use is refused where it reads it,** by the point's own
``Projection`` cell, naming the point, never here: no point is refused for another's
value, nor for a column its projection never reads, so that the model projects every
point ``SPIA_US_S`` projects to a number, as at stage ``E`` (D2). Where a vector cannot hold
a value, the vector holds 0 or NaN there and its ``_ok`` flag 0, and the
:mod:`~.SPIA_US_SEC.Projection` accessor that reads the element raises:

* text in a column of numbers, which ``SPIA_US_S``'s ``float()`` or ``int()`` refuses in
  the same cell, on every point that reads it;
* a blank, which ``int()`` refuses, in an age, a month count, a frequency or a year, as
  in ``SPIA_US_S``: a blank ``joint_age`` on a single-life point, a blank
  ``certain_months`` on a form without an elected period and a blank ``annuity_year`` on
  the scenario basis are read by no projection, and refused only by a direct call;
* an int of ``2**53`` or more in size, which an int64 cannot be trusted to hold
  (below), in those columns and in ``duration_mth`` and the two death months;
* a blank or infinite amount or rate on a point whose projection reads it through a
  ``max``, a ``min`` or a comparison that would turn the missing value into a number, as
  ``SPIA_US_S``'s would: the premium on a cash or installment refund, whose refund
  balance is ``max(0, P - G)`` and whose refund period a search ``G(k) >= P``; the first
  year's income and the COLA rate on every point, whose instalments every projection
  pays and its refund, its refund period and its withdrawal's floor read through such a
  ``max``, ``min`` or search; the survivor percentage on a joint contract of the four
  forms with a life contingency, whose payment factor the certain floor's ``max`` reads;
  and the rating factor on the table basis, whose mortality the same ``max`` reads in the
  end.
  Elsewhere a blank stays NaN, as ``SPIA_US_S`` reads it, and so does every blank
  ``premium_tax_rate``, which only ``Projection.premium_net_pp`` reads, in a product.

That is the family's cap rule (USLIB-SEC-PLAN.md §7.1), applied by point: a point
``SPIA_US_S`` projects to a number only because a ``max`` or a ``min`` lost its missing
value is refused, in its projection, and no other. A fraction in an int column truncates,
as ``SPIA_US_S``'s ``int()`` does, so a frequency written 12.5 is monthly. A text value of
``form``, ``reduction_trigger``, ``timing`` or ``mort_basis`` that is none of
``SPIA_US_S``'s values sets no ``has_*`` flag, and the decode that reads the flag raises
``SPIA_US_S``'s message there, as at stage ``E``; the values are compared exactly, case and
whitespace included, as ``SPIA_US_S`` compares them. A flag column is ``bool()`` of each
element as pandas reads it, as ``SPIA_US_S``'s ``bool(model_point()[...])`` is, so a blank
reads as True there too. What the table's own checks refuse, :func:`model_point_table`
refuses for the whole table, as at stage ``E``, and :func:`point_id` refuses a label that
is not a whole number below ``2**53``, naming every label at fault, when a label is first
read: they are the table's keys and its in-force interface.

**An int64 holds what it is read from.** A float past int64 casts to the least int64
without a word, so a number read into an int64 vector is usable only below ``2**53`` in
size, where the int64 holds ``SPIA_US_S``'s reading of it exactly; past it the flag is 0.

**The large extract.** :func:`model_point_table` reads the file whole, not in chunks
(``low_memory=False``), so that pandas infers each column's type once: read in chunks, a
table of thousands of single lives with a joint point among them typed ``joint_sex`` chunk
by chunk and warned (``DtypeWarning``), though the frame was the same.
"""

from modelx.serialize.jsonvalues import *

_formula = None

_bases = []

_allow_none = None

_spaces = []

# ---------------------------------------------------------------------------
# Cells

[docs] def input_dir(): """The directory holding the input CSVs: the model folder's parent. Inputs are *external* files, not data stored inside the model, so the model folder is pure formulas. The path is resolved at run time from where the model was read, following ``annuallife.TradLife_A``. """ return _model.path.parent # noqa: F821
[docs] def model_point_table(): """The model point table, read from *model_point_table_sec.csv* and checked. Indexed by ``point_id``. Six columns are the in-force interface: ``policy_count``, the contracts the point stands for, and ``duration_mth``, the months elapsed from the annuity date at the valuation date, as in ``basiclife.BasicTerm_SE``; then the opening state, per contract, as it stands at the start of month 0, before any month-0 event: * ``lives_if_init_1`` and ``lives_if_init_2``: each covered life's probability of being alive at the valuation date, 1 or 0 on a real record, where a joint contract may carry one life already dead, and 0 for the second life of a single-life contract; * ``commute_frac_cum_init``: the cumulative commutation fraction applying in month 0, before any month-0 withdrawal; * ``cmt10_shift_init``: the 10-year CMT at the valuation date less the 10-year CMT at the annuity date, as a decimal, which moves the commutation discount rate. The rest are ``SPIA_US_S``'s, ``pols_if_init`` renamed ``policy_count``, except ``row_kind``, ``source_policy_id`` and ``source_mth``, which record where the generator took each shipped row from and which no formula reads. The checks run over the whole table at once, and the first that fails raises ``ValueError``, naming the missing column or every point it fails on: * the six in-force columns are present; * no ``point_id`` appears twice; * ``policy_count`` is a number, zero or more; * ``duration_mth`` is a whole number of months, read by value, so that ``12.0`` is 12: positive for a contract in payment, 0 for one whose annuity date is the valuation date, negative for one commencing later; * no annuity commences more than ``future_horizon_mth`` months after the valuation date, so ``duration_mth`` is ``-future_horizon_mth`` or more. At 0 the model takes no future new business, and the message says so; a negative horizon is itself refused; * ``lives_if_init_1`` and ``lives_if_init_2`` are probabilities, 0 to 1; * ``commute_frac_cum_init`` is a number, 0 or more and below 1; * ``cmt10_shift_init`` is a number, of either sign; * a single-life contract has no second life: ``lives_if_init_2`` is 0; * a contract not yet in payment, ``duration_mth`` 0 or less, holds its values at the annuity date: each covered life 1, no commutation and no change in the CMT; * each month of death, ``death_mth_1`` and ``death_mth_2``, is blank or a whole number of months, read by value, so that ``13.0`` is 13 and a fraction or text is refused, on every basis. The table is returned with the two columns as the numbers they were checked as, which is what ``Projection.death_mth`` reads; * on the scenario basis, each life's opening status is its death month's: 1 while the life survives to the valuation date, ``death_mth`` blank, negative or ``duration_mth`` or later, as ``Projection.death_mth`` and ``Projection.lives_if`` read it, and 0 once it has died. So the scenario's step function, which ``lives_if`` follows after the valuation date, agrees with the opening value at it; * a commutation fraction appears only where a commutation could have been taken before the valuation date: the contract holds the right, its issue state allows it, it has a certain period, an elected one or an installment refund's, and ``duration_mth`` is 13 or more, the first withdrawal being taken in policy month 12 and reducing the instalments from month 13. That is a necessary condition only: the check cannot see an installment refund's derived period, which needs the projection, nor whether a certain period long enough for a withdrawal remained, and it does not ask whether the model's own utilization assumption would have taken one, since a real record's withdrawals do not depend on it. Whether the contract still owes a payment at the valuation date needs the certain period, which an installment refund derives in ``Projection``, so ``Projection.proj_len`` checks it. The file is read whole, ``low_memory=False``, so that pandas types each column once, not chunk by chunk, which on a large extract warned and changed nothing. The two death columns are returned through ``**{...}``, since their names are cells of this Space, which the export would rewrite as keywords. """ name = model_point_file # noqa: F821 table = pd.read_csv(input_dir() / name, index_col="point_id", # noqa: F821 low_memory=False) state = ("lives_if_init_1", "lives_if_init_2", "commute_frac_cum_init", "cmt10_shift_init") for column in ("policy_count", "duration_mth", *state): if column not in table.columns: raise ValueError(f"{name} has no {column} column") repeated = table.index[table.index.duplicated()].unique().tolist() if repeated: raise ValueError(f"{name}: point_id {repeated} appears more than once") count = pd.to_numeric(table["policy_count"], errors="coerce") # noqa: F821 bad = table.index[~((count >= 0) & (count < float("inf")))].tolist() if bad: raise ValueError(f"{name}: policy_count must be a number, zero or more; it is " f"not at point_id {bad}") months = pd.to_numeric(table["duration_mth"], errors="coerce") # noqa: F821 bad = table.index[~(months % 1 == 0)].tolist() if bad: raise ValueError(f"{name}: duration_mth must be a whole number of months; it " f"is not at point_id {bad}") horizon = future_horizon_mth # noqa: F821 if horizon < 0: raise ValueError(f"future_horizon_mth must be 0 months or more, not {horizon}") bad = table.index[months < -horizon].tolist() if bad and horizon == 0: raise ValueError(f"{name}: duration_mth must be 0 or more: this model takes no " f"future new business; it is not at point_id {bad}") if bad: raise ValueError(f"{name}: duration_mth must be -{horizon} or more, a policy " f"issued at most {horizon} months after the valuation date; it " f"is not at point_id {bad}") values = {c: pd.to_numeric(table[c], errors="coerce") for c in state} # noqa: F821 for column in ("lives_if_init_1", "lives_if_init_2"): v = values[column] bad = table.index[~((v >= 0) & (v <= 1))].tolist() if bad: raise ValueError(f"{name}: {column} must be a probability, 0 to 1; it is " f"not at point_id {bad}") theta = values["commute_frac_cum_init"] bad = table.index[~((theta >= 0) & (theta < 1))].tolist() if bad: raise ValueError(f"{name}: commute_frac_cum_init must be a number, 0 or more " f"and below 1; it is not at point_id {bad}") shift = values["cmt10_shift_init"] bad = table.index[~((shift > -float("inf")) & (shift < float("inf")))].tolist() if bad: raise ValueError(f"{name}: cmt10_shift_init must be a number, the change in the " f"10-year CMT since the annuity date; it is not at point_id {bad}") joint = table["joint"].astype(bool) bad = table.index[~joint & (values["lives_if_init_2"] != 0)].tolist() if bad: raise ValueError(f"{name}: lives_if_init_2 must be 0 on a single-life " f"contract, which covers no second life; it is not at " f"point_id {bad}") issued = months <= 0 for column, at_issue in (("lives_if_init_1", 1.0), ("lives_if_init_2", joint.astype(float)), ("commute_frac_cum_init", 0.0), ("cmt10_shift_init", 0.0)): bad = table.index[issued & (values[column] != at_issue)].tolist() if bad: raise ValueError(f"{name}: {column} must hold its value at issue on a " f"contract not yet in force, duration_mth 0 or less: each " f"covered life alive, no commutation taken and no change in " f"the CMT; it does not at point_id {bad}") deaths = {} for life in (1, 2): column = f"death_mth_{life}" death = pd.to_numeric(table[column], errors="coerce") # noqa: F821 bad = table.index[~(table[column].isna() | (death % 1 == 0))].tolist() if bad: raise ValueError(f"{name}: {column} must be blank or a whole number of " f"months, a policy month; it is not at point_id {bad}") deaths[life] = death scenario = table["mort_basis"] == "scenario" for life in (1, 2): death = deaths[life] alive = death.isna() | (death < 0) | (death >= months) covered = joint if life == 2 else pd.Series(True, index=table.index) # noqa: F821 want = alive.astype(float).where(covered, 0.0) column = f"lives_if_init_{life}" bad = table.index[scenario & (values[column] != want)].tolist() if bad: raise ValueError(f"{name}: on the scenario basis {column} must be 1 while " f"the life survives to the valuation date, death_mth_{life} " f"blank, negative or duration_mth or later, and 0 once it " f"has died; it is not at point_id {bad}") form = table["form"] certain = (form.isin(["life_certain", "certain_only"]) & (pd.to_numeric(table["certain_months"], errors="coerce") > 0) # noqa: F821 | (form == "installment_refund")) may = (table["commutation_enabled"].astype(bool) & ~table["issue_state_excludes_withdrawal"].astype(bool) & certain & (months >= 13)) bad = table.index[(theta > 0) & ~may].tolist() if bad: raise ValueError(f"{name}: commute_frac_cum_init must be 0 unless a commutation " f"could have been taken before the valuation date: the right " f"held, the issue state allowing it, a certain period, and " f"duration_mth 13 or more; it is not at point_id {bad}") return table.assign(**{"death_mth_1": deaths[1], "death_mth_2": deaths[2]})
[docs] def mort_table(): """The base annuitant mortality table by age and sex, read from *mort_table.csv*.""" return pd.read_csv( # noqa: F821 input_dir() / mort_table_file, index_col=["age", "sex"]) # noqa: F821
[docs] def improvement_scale(): """The generational improvement scale by age and sex, from *improvement_scale.csv*.""" return pd.read_csv( # noqa: F821 input_dir() / improvement_scale_file, index_col=["age", "sex"]) # noqa: F821
[docs] def surr_charge_table(): """The commutation surrender-charge scale by contract year, *surr_charge_table.csv*.""" return pd.read_csv( # noqa: F821 input_dir() / surr_charge_file, index_col="policy_year") # noqa: F821
[docs] def surr_charge_scale(): """The surrender-charge scale by contract year, as :func:`surr_charge_table` reads it, checked. The rates of *surr_charge_table.csv*, indexed by its policy years read by value, as ``SPIA_US_S``'s lookup matches them, so that a year written ``2.0`` is year 2. ``Projection.surr_charge_rate`` takes the scale's first and last years and its rates from here, so the extents and the values come from one checked column. A year that is not a whole number, text and a blank among them, is refused, naming every such year; so is a scale with no year, or whose years do not run from the first to the last without a gap or a repeat, from year 1 on, each with a rate. ``SPIA_US_S`` takes ``int()`` of the raw years for the extents, so a year written ``x`` raises Python's own message there, and one written ``3a`` turns the years to text and makes every lookup miss. """ file = surr_charge_file # noqa: F821 tbl = surr_charge_table() if len(tbl) == 0: raise ValueError(f"{file} has no policy year") years = pd.to_numeric(pd.Series(tbl.index), errors="coerce") # noqa: F821 bad = tbl.index[~(years % 1 == 0).to_numpy()].tolist() if bad: raise ValueError(f"{file}: a policy year must be a whole number; {bad} is not") whole = years.astype("int64").tolist() first, last = min(whole), max(whole) rates = pd.to_numeric(tbl["surr_charge_rate"], errors="coerce") # noqa: F821 if (first < 1 or sorted(whole) != list(range(first, last + 1)) or rates.isna().any()): raise ValueError(f"{file}: the policy years must run from {first} to {last} " f"without a gap or a repeat, from 1 on, each with a rate") return pd.DataFrame( # noqa: F821 {"surr_charge_rate": rates.to_numpy(dtype="float64")}, index=pd.Index(whole, name="policy_year")) # noqa: F821
[docs] def point_id(): """The model points' labels, int64 by position: the table's ``point_id`` index. The label ``Projection.point_id`` reports for a position, and the one every refusal names. Read by value, so that a label written ``3.0`` is 3, and refused when a label is first read, naming every label at fault, where one is not a whole number, text and a blank among them, or is ``2**53`` or more in size: the labels are the table's keys, as in the family's other compiled models. A projection that names no point reads no label, and runs. """ table = model_point_table() labels = pd.to_numeric(pd.Series(table.index), errors="coerce") # noqa: F821 bad = table.index[~(labels % 1 == 0).to_numpy()].tolist() if bad: raise ValueError(f"{model_point_file}: point_id must be a whole " # noqa: F821 f"number; {bad} is not") bad = table.index[~(labels.abs() < 2.0 ** 53).to_numpy()].tolist() if bad: raise ValueError(f"{model_point_file}: point_id must be below 2**53 " # noqa: F821 f"in size; {bad} is not") return labels.to_numpy(dtype="int64")
[docs] def policy_count(): """The ``policy_count`` column, float64 by point, as :func:`model_point_table` checked it.""" return model_point_table()["policy_count"].to_numpy(dtype="float64")
[docs] def duration_mth(): """The ``duration_mth`` column, int64 by point: ``int()`` of each value, 0 where :func:`duration_mth_ok` is 0. :func:`model_point_table` has refused a value that is not a whole number, read by value, so that ``12.0`` is 12. """ values = model_point_table()["duration_mth"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = v return out
[docs] def duration_mth_ok(): """1 where ``int()`` reads the point's ``duration_mth`` below ``2**53`` in size, as :func:`primary_age_ok`: 0 on a blank, and on text, ``"12.0"`` among them in a column pandas reads as text, which ``int()`` refuses; a number read as one, ``12.0``, is 12.""" values = model_point_table()["duration_mth"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = 1 return out
[docs] def premium(): """The ``premium`` column, float64 by point: ``float()`` of each value, NaN on text.""" values = model_point_table()["premium"].tolist() out = np.full(len(values), np.nan) # noqa: F821 for i in range(len(values)): try: out[i] = float(values[i]) except (TypeError, ValueError): pass return out
[docs] def premium_ok(): """1 where the point's projection may read its ``premium``. 0 on text, which ``float()`` refuses, and on a blank or infinite premium on a cash or installment refund, whose refund balance ``max(0, P - G)`` and refund period ``G(k) >= P`` would turn it into a number. A blank stays NaN on the other forms, which read it only in ``Projection.premium_net_pp``, as ``SPIA_US_S`` does. """ table = model_point_table() values = table["premium"].tolist() forms = table["form"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = float(values[i]) except (TypeError, ValueError): continue if abs(v) < float("inf") or not (forms[i] == "cash_refund" or forms[i] == "installment_refund"): out[i] = 1 return out
[docs] def premium_tax_rate(): """The ``premium_tax_rate`` column, float64 by point: ``float()`` of each value, NaN on text.""" values = model_point_table()["premium_tax_rate"].tolist() out = np.full(len(values), np.nan) # noqa: F821 for i in range(len(values)): try: out[i] = float(values[i]) except (TypeError, ValueError): pass return out
[docs] def premium_tax_rate_ok(): """1 where ``float()`` reads the point's ``premium_tax_rate``: 0 on text alone. A blank stays NaN, as ``SPIA_US_S`` reads it: only ``Projection.premium_net_pp``, which no report reaches, reads the rate. """ values = model_point_table()["premium_tax_rate"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: float(values[i]) except (TypeError, ValueError): continue out[i] = 1 return out
[docs] def annual_income(): """The ``annual_income`` column, B(1), float64 by point: ``float()`` of each value, NaN on text.""" values = model_point_table()["annual_income"].tolist() out = np.full(len(values), np.nan) # noqa: F821 for i in range(len(values)): try: out[i] = float(values[i]) except (TypeError, ValueError): pass return out
[docs] def annual_income_ok(): """1 where the point's ``annual_income`` is a finite number: 0 on text, a blank and an infinity. Every projection pays its instalments, and its refund, its refund period and its withdrawal's residual floor read them through a ``max``, a ``min`` or a search that would turn a missing income into a number. """ values = model_point_table()["annual_income"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = float(values[i]) except (TypeError, ValueError): continue if abs(v) < float("inf"): out[i] = 1 return out
[docs] def cola_rate(): """The ``cola_rate`` column, float64 by point: ``float()`` of each value, NaN on text.""" values = model_point_table()["cola_rate"].tolist() out = np.full(len(values), np.nan) # noqa: F821 for i in range(len(values)): try: out[i] = float(values[i]) except (TypeError, ValueError): pass return out
[docs] def cola_rate_ok(): """1 where the point's ``cola_rate`` is a finite number: 0 on text, a blank and an infinity. It compounds every instalment past the first policy year, which the projection reads as :func:`annual_income_ok` says. """ values = model_point_table()["cola_rate"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = float(values[i]) except (TypeError, ValueError): continue if abs(v) < float("inf"): out[i] = 1 return out
[docs] def survivor_pct(): """The ``survivor_pct`` column, float64 by point: ``float()`` of each value, NaN on text.""" values = model_point_table()["survivor_pct"].tolist() out = np.full(len(values), np.nan) # noqa: F821 for i in range(len(values)): try: out[i] = float(values[i]) except (TypeError, ValueError): pass return out
[docs] def survivor_pct_ok(): """1 where the point's projection may read its ``survivor_pct``. 0 on text, and on a blank or infinite percentage on a joint contract of the four forms with a life contingency, whose payment factor, ``max(C, L)``, would turn a missing one into a number. A single-life contract and a certain_only one never read it. """ table = model_point_table() values = table["survivor_pct"].tolist() joint = table["joint"].tolist() forms = table["form"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = float(values[i]) except (TypeError, ValueError): continue if (abs(v) < float("inf") or not bool(joint[i]) or forms[i] == "certain_only"): out[i] = 1 return out
[docs] def rating_factor(): """The ``rating_factor`` column, float64 by point: ``float()`` of each value, NaN on text.""" values = model_point_table()["rating_factor"].tolist() out = np.full(len(values), np.nan) # noqa: F821 for i in range(len(values)): try: out[i] = float(values[i]) except (TypeError, ValueError): pass return out
[docs] def rating_factor_ok(): """1 where the point's projection may read its ``rating_factor``. 0 on text, and on a blank or infinite factor on the table basis, whose survival the payment factor's ``max(C, L)`` reads in the end: the mortality caps put their bound second, so a missing rating stays missing there, and the ``max`` would turn it into a number. The scenario basis never reads it. """ table = model_point_table() values = table["rating_factor"].tolist() basis = table["mort_basis"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = float(values[i]) except (TypeError, ValueError): continue if abs(v) < float("inf") or not basis[i] == "table": out[i] = 1 return out
[docs] def primary_age(): """The ``primary_age`` column, int64 by point: ``int()`` of each value, 0 where :func:`primary_age_ok` is 0.""" values = model_point_table()["primary_age"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = v return out
[docs] def primary_age_ok(): """1 where ``int()`` reads the point's ``primary_age`` below ``2**53`` in size. 0 on text, a blank and an infinity, which ``SPIA_US_S``'s ``int()`` refuses, and on a number the int64 vector cannot be trusted to hold. A fraction truncates, as ``int()`` does. """ values = model_point_table()["primary_age"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = 1 return out
[docs] def joint_age(): """The ``joint_age`` column, int64 by point: ``int()`` of each value, 0 where :func:`joint_age_ok` is 0, a single-life point's blank among them.""" values = model_point_table()["joint_age"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = v return out
[docs] def joint_age_ok(): """1 where ``int()`` reads the point's ``joint_age`` below ``2**53`` in size. As :func:`primary_age_ok`. A single-life point's blank is 0 here, and no projection of such a point reads it. """ values = model_point_table()["joint_age"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = 1 return out
[docs] def certain_months(): """The ``certain_months`` column, int64 by point: ``int()`` of each value, 0 where :func:`certain_months_ok` is 0.""" values = model_point_table()["certain_months"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = v return out
[docs] def certain_months_ok(): """1 where ``int()`` reads the point's ``certain_months`` below ``2**53`` in size. As :func:`primary_age_ok`. Only a form with an elected certain period reads it. """ values = model_point_table()["certain_months"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = 1 return out
[docs] def frequency(): """The ``frequency`` column, int64 by point: ``int()`` of each value, 0 where :func:`frequency_ok` is 0. Any whole number is held; ``Projection.payment_freq`` refuses one that is not 12, 4, 2 or 1, with ``SPIA_US_S``'s message. """ values = model_point_table()["frequency"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = v return out
[docs] def frequency_ok(): """1 where ``int()`` reads the point's ``frequency`` below ``2**53`` in size, as :func:`primary_age_ok`.""" values = model_point_table()["frequency"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = 1 return out
[docs] def annuity_year(): """The ``annuity_year`` column, int64 by point: ``int()`` of each value, 0 where :func:`annuity_year_ok` is 0.""" values = model_point_table()["annuity_year"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = v return out
[docs] def annuity_year_ok(): """1 where ``int()`` reads the point's ``annuity_year`` below ``2**53`` in size. As :func:`primary_age_ok`. Only the table basis reads it, to date improvement. """ values = model_point_table()["annuity_year"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): try: v = int(values[i]) except (TypeError, ValueError, OverflowError): continue if abs(v) < 2 ** 53: out[i] = 1 return out
[docs] def is_joint(): """The ``joint`` column, int64 by point: 1 where ``bool()`` of the value is True.""" values = model_point_table()["joint"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): out[i] = 1 if bool(values[i]) else 0 return out
[docs] def commutation_enabled(): """The ``commutation_enabled`` column, int64 by point: 1 where ``bool()`` of the value is True.""" values = model_point_table()["commutation_enabled"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): out[i] = 1 if bool(values[i]) else 0 return out
[docs] def issue_state_excludes_withdrawal(): """The ``issue_state_excludes_withdrawal`` column, int64 by point: 1 where ``bool()`` of the value is True.""" values = model_point_table()["issue_state_excludes_withdrawal"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): out[i] = 1 if bool(values[i]) else 0 return out
[docs] def qualified(): """The ``qualified`` column, int64 by point: 1 where ``bool()`` of the value is True.""" values = model_point_table()["qualified"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): out[i] = 1 if bool(values[i]) else 0 return out
[docs] def has_form(): """1 where the point's ``form`` is one of ``SPIA_US_S``'s five, compared exactly.""" f = model_point_table()["form"] return f.isin(["life_only", "life_certain", "cash_refund", "installment_refund", "certain_only"]).to_numpy(dtype="int64")
[docs] def is_period_certain(): """1 where the point's ``form`` is ``life_certain`` or ``certain_only``.""" f = model_point_table()["form"] return ((f == "life_certain") | (f == "certain_only")).to_numpy(dtype="int64")
[docs] def is_certain_only(): """1 where the point's ``form`` is ``certain_only``.""" return (model_point_table()["form"] == "certain_only").to_numpy(dtype="int64")
[docs] def is_cash_refund(): """1 where the point's ``form`` is ``cash_refund``.""" return (model_point_table()["form"] == "cash_refund").to_numpy(dtype="int64")
[docs] def is_installment_refund(): """1 where the point's ``form`` is ``installment_refund``.""" return (model_point_table()["form"] == "installment_refund").to_numpy(dtype="int64")
[docs] def has_reduction_trigger(): """1 where the point's ``reduction_trigger`` is ``either`` or ``primary``.""" return model_point_table()["reduction_trigger"].isin( ["either", "primary"]).to_numpy(dtype="int64")
[docs] def is_trigger_either(): """1 where the point's ``reduction_trigger`` is ``either``.""" return (model_point_table()["reduction_trigger"] == "either").to_numpy(dtype="int64")
[docs] def has_timing(): """1 where the point's ``timing`` is ``arrears`` or ``advance``.""" return model_point_table()["timing"].isin(["arrears", "advance"]).to_numpy( dtype="int64")
[docs] def is_arrears(): """1 where the point's ``timing`` is ``arrears``.""" return (model_point_table()["timing"] == "arrears").to_numpy(dtype="int64")
[docs] def has_mort_basis(): """1 where the point's ``mort_basis`` is ``table`` or ``scenario``.""" return model_point_table()["mort_basis"].isin(["table", "scenario"]).to_numpy( dtype="int64")
[docs] def is_scenario(): """1 where the point's ``mort_basis`` is ``scenario``.""" return (model_point_table()["mort_basis"] == "scenario").to_numpy(dtype="int64")
[docs] def death_mth_1(): """The primary's scenario month of death, int64 by point: -1 where blank. The ``death_mth_1`` column as :func:`model_point_table` checked it, a whole number of policy months or blank, the blank being ``SPIA_US_S``'s survivor, which ``Projection.death_mth`` reads as -1; 0 where :func:`death_mth_1_ok` is 0. """ values = model_point_table()["death_mth_1"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): v = values[i] if not v == v: out[i] = -1 elif abs(v) < 2 ** 53: out[i] = int(v) return out
[docs] def death_mth_1_ok(): """1 where :func:`death_mth_1` holds the point's month: blank, or below ``2**53``.""" values = model_point_table()["death_mth_1"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): v = values[i] if not v == v or abs(v) < 2 ** 53: out[i] = 1 return out
[docs] def death_mth_2(): """The joint annuitant's scenario month of death, int64 by point: -1 where blank, as :func:`death_mth_1`.""" values = model_point_table()["death_mth_2"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): v = values[i] if not v == v: out[i] = -1 elif abs(v) < 2 ** 53: out[i] = int(v) return out
[docs] def death_mth_2_ok(): """1 where :func:`death_mth_2` holds the point's month: blank, or below ``2**53``.""" values = model_point_table()["death_mth_2"].tolist() out = np.zeros(len(values), dtype="int64") # noqa: F821 for i in range(len(values)): v = values[i] if not v == v or abs(v) < 2 ** 53: out[i] = 1 return out
[docs] def lives_if_init_1(): """The ``lives_if_init_1`` column, float64 by point: checked by :func:`model_point_table`.""" return pd.to_numeric(model_point_table()["lives_if_init_1"]).to_numpy( # noqa: F821 dtype="float64")
[docs] def lives_if_init_2(): """The ``lives_if_init_2`` column, float64 by point: checked by :func:`model_point_table`.""" return pd.to_numeric(model_point_table()["lives_if_init_2"]).to_numpy( # noqa: F821 dtype="float64")
[docs] def commute_frac_cum_init(): """The ``commute_frac_cum_init`` column, float64 by point: checked by :func:`model_point_table`.""" return pd.to_numeric(model_point_table()["commute_frac_cum_init"]).to_numpy( # noqa dtype="float64")
[docs] def cmt10_shift_init(): """The ``cmt10_shift_init`` column, float64 by point: checked by :func:`model_point_table`.""" return pd.to_numeric(model_point_table()["cmt10_shift_init"]).to_numpy( # noqa: F821 dtype="float64")
[docs] def mort_sexes(): """*mort_table.csv*'s check, an int: the number of its sexes, each a schedule. The table is read by value, as ``SPIA_US_S``'s ``.loc`` matches its keys, and refused, naming the file and every line at fault, where an age is not a whole number, text and a blank among them, a sex is blank, or a rate is not a finite number; and, naming the sex, where one sex's ages do not run from its first to its last without a gap or a repeat, or one is negative, since the table is held from age 0. ``SPIA_US_S`` raised ``KeyError`` only where a projection read a missing key, and read a blank rate as NaN. Every array, extent and schedule of the table reads this first. """ name = mort_table_file # noqa: F821 rows = mort_table().reset_index() ages = pd.to_numeric(rows["age"], errors="coerce") # noqa: F821 bad = (rows.index[~(ages % 1 == 0)] + 2).tolist() if bad: raise ValueError(f"{name}: age must be a whole number; it is not on line {bad}") bad = (rows.index[rows["sex"].isna()] + 2).tolist() if bad: raise ValueError(f"{name}: sex must not be blank; it is on line {bad}") rates = pd.to_numeric(rows["mort_rate"], errors="coerce") # noqa: F821 bad = (rows.index[~(rates.abs() < float("inf"))] + 2).tolist() if bad: raise ValueError(f"{name}: mort_rate must be a finite number; it is not on line " f"{bad}") rows = rows.assign(age=ages.astype("int64")) for sex, part in rows.groupby("sex", sort=True): a = part["age"].tolist() if min(a) < 0 or sorted(a) != list(range(min(a), max(a) + 1)): raise ValueError(f"{name}: the ages of sex {sex} must run from {min(a)} to " f"{max(a)} without a gap or a repeat, none negative") return int(rows["sex"].nunique())
[docs] def mort_table_array(): """*mort_table.csv*'s rates, float64 by schedule and attained age, ``[s, x]``. Schedule ``s`` is the ``s``-th of the table's sexes in sorted order, from 1; row 0, and every age below a sex's first, is NaN padding, which ``Projection.mort_rate_base_at`` never reads, since it checks the age against :func:`mort_age_first` and :func:`mort_age_last` first. """ n = mort_sexes() rows = mort_table().reset_index() rows = rows.assign(age=pd.to_numeric(rows["age"]).astype("int64")) # noqa: F821 out = np.full((n + 1, int(rows["age"].max()) + 1), np.nan) # noqa: F821 s = 0 for _, part in rows.groupby("sex", sort=True): s += 1 out[s, part["age"].to_numpy()] = pd.to_numeric( # noqa: F821 part["mort_rate"]).to_numpy(dtype="float64") return out
[docs] def mort_age_first(): """Each mortality schedule's first age, int64 by schedule: 1 on schedule 0.""" out = np.ones(mort_sexes() + 1, dtype="int64") # noqa: F821 rows = mort_table().reset_index() rows = rows.assign(age=pd.to_numeric(rows["age"]).astype("int64")) # noqa: F821 s = 0 for _, part in rows.groupby("sex", sort=True): s += 1 out[s] = int(part["age"].min()) return out
[docs] def mort_age_last(): """Each mortality schedule's last age, int64 by schedule: 0 on schedule 0, so that no age lies between its extents.""" out = np.zeros(mort_sexes() + 1, dtype="int64") # noqa: F821 rows = mort_table().reset_index() rows = rows.assign(age=pd.to_numeric(rows["age"]).astype("int64")) # noqa: F821 s = 0 for _, part in rows.groupby("sex", sort=True): s += 1 out[s] = int(part["age"].max()) return out
[docs] def mort_schedule_1(): """The mortality schedule of each point's ``primary_sex``, int64 by point. The sex's number in :func:`mort_table_array`, matched as ``SPIA_US_S``'s ``.loc`` matches it, and 0 where the table lacks it, a blank among them: a point on the scenario basis never reads the table, and one on the table basis is refused where ``Projection.mort_rate_base_at`` first reads schedule 0. """ mort_sexes() keys = sorted(mort_table().reset_index()["sex"].unique()) number = {} for i in range(len(keys)): number[keys[i]] = i + 1 sexes = model_point_table()["primary_sex"].tolist() out = np.zeros(len(sexes), dtype="int64") # noqa: F821 for i in range(len(sexes)): out[i] = number.get(sexes[i], 0) return out
[docs] def mort_schedule_2(): """The mortality schedule of each point's ``joint_sex``, int64 by point, as :func:`mort_schedule_1`: 0 on a single-life point's blank, which nothing reads.""" mort_sexes() keys = sorted(mort_table().reset_index()["sex"].unique()) number = {} for i in range(len(keys)): number[keys[i]] = i + 1 sexes = model_point_table()["joint_sex"].tolist() out = np.zeros(len(sexes), dtype="int64") # noqa: F821 for i in range(len(sexes)): out[i] = number.get(sexes[i], 0) return out
[docs] def improvement_sexes(): """*improvement_scale.csv*'s check, an int: the number of its sexes, each a schedule. The scale is read and refused as :func:`mort_sexes` reads and refuses the mortality table, and numbered by its own sexes: the two tables need not hold the same ones. """ name = improvement_scale_file # noqa: F821 rows = improvement_scale().reset_index() ages = pd.to_numeric(rows["age"], errors="coerce") # noqa: F821 bad = (rows.index[~(ages % 1 == 0)] + 2).tolist() if bad: raise ValueError(f"{name}: age must be a whole number; it is not on line {bad}") bad = (rows.index[rows["sex"].isna()] + 2).tolist() if bad: raise ValueError(f"{name}: sex must not be blank; it is on line {bad}") rates = pd.to_numeric(rows["improvement_rate"], errors="coerce") # noqa: F821 bad = (rows.index[~(rates.abs() < float("inf"))] + 2).tolist() if bad: raise ValueError(f"{name}: improvement_rate must be a finite number; it is not " f"on line {bad}") rows = rows.assign(age=ages.astype("int64")) for sex, part in rows.groupby("sex", sort=True): a = part["age"].tolist() if min(a) < 0 or sorted(a) != list(range(min(a), max(a) + 1)): raise ValueError(f"{name}: the ages of sex {sex} must run from {min(a)} to " f"{max(a)} without a gap or a repeat, none negative") return int(rows["sex"].nunique())
[docs] def improvement_array(): """*improvement_scale.csv*'s rates, float64 by schedule and attained age, ``[s, x]``, padded as :func:`mort_table_array` is.""" n = improvement_sexes() rows = improvement_scale().reset_index() rows = rows.assign(age=pd.to_numeric(rows["age"]).astype("int64")) # noqa: F821 out = np.full((n + 1, int(rows["age"].max()) + 1), np.nan) # noqa: F821 s = 0 for _, part in rows.groupby("sex", sort=True): s += 1 out[s, part["age"].to_numpy()] = pd.to_numeric( # noqa: F821 part["improvement_rate"]).to_numpy(dtype="float64") return out
[docs] def improvement_age_first(): """Each improvement schedule's first age, int64 by schedule: 1 on schedule 0.""" out = np.ones(improvement_sexes() + 1, dtype="int64") # noqa: F821 rows = improvement_scale().reset_index() rows = rows.assign(age=pd.to_numeric(rows["age"]).astype("int64")) # noqa: F821 s = 0 for _, part in rows.groupby("sex", sort=True): s += 1 out[s] = int(part["age"].min()) return out
[docs] def improvement_age_last(): """Each improvement schedule's last age, int64 by schedule: 0 on schedule 0.""" out = np.zeros(improvement_sexes() + 1, dtype="int64") # noqa: F821 rows = improvement_scale().reset_index() rows = rows.assign(age=pd.to_numeric(rows["age"]).astype("int64")) # noqa: F821 s = 0 for _, part in rows.groupby("sex", sort=True): s += 1 out[s] = int(part["age"].max()) return out
[docs] def improvement_schedule_1(): """The improvement schedule of each point's ``primary_sex``, int64 by point, numbered by the scale's own sexes, as :func:`mort_schedule_1` by the table's.""" improvement_sexes() keys = sorted(improvement_scale().reset_index()["sex"].unique()) number = {} for i in range(len(keys)): number[keys[i]] = i + 1 sexes = model_point_table()["primary_sex"].tolist() out = np.zeros(len(sexes), dtype="int64") # noqa: F821 for i in range(len(sexes)): out[i] = number.get(sexes[i], 0) return out
[docs] def improvement_schedule_2(): """The improvement schedule of each point's ``joint_sex``, int64 by point, as :func:`improvement_schedule_1`.""" improvement_sexes() keys = sorted(improvement_scale().reset_index()["sex"].unique()) number = {} for i in range(len(keys)): number[keys[i]] = i + 1 sexes = model_point_table()["joint_sex"].tolist() out = np.zeros(len(sexes), dtype="int64") # noqa: F821 for i in range(len(sexes)): out[i] = number.get(sexes[i], 0) return out
[docs] def surr_charge_rate_array(): """The surrender-charge rates, float64 by policy year, from :func:`surr_charge_scale`. Indexed by the year itself, from 0, NaN below the scale's first year, which ``Projection.surr_charge_rate`` never reads: it clamps the year to :func:`surr_charge_year_first` and :func:`surr_charge_year_last` first, and the scale's check holds the first year to 1 or more. """ scale = surr_charge_scale() out = np.full(int(scale.index.max()) + 1, np.nan) # noqa: F821 out[scale.index.to_numpy()] = scale["surr_charge_rate"].to_numpy(dtype="float64") return out
[docs] def surr_charge_year_first(): """The surrender-charge scale's first policy year, an int, from :func:`surr_charge_scale`.""" return int(surr_charge_scale().index.min())
[docs] def surr_charge_year_last(): """The surrender-charge scale's last policy year, an int, from :func:`surr_charge_scale`.""" return int(surr_charge_scale().index.max())
# --------------------------------------------------------------------------- # References model_point_file = "model_point_table_sec.csv" mort_table_file = "mort_table.csv" improvement_scale_file = "improvement_scale.csv" surr_charge_file = "surr_charge_table.csv" future_horizon_mth = 120 pd = ("Module", "pandas") np = ("Module", "numpy")