ADM Explained

Pega

2023-03-15

This notebook shows exactly how all the values in an ADM model report are calculated. It also shows how the propensity is calculated for a particular customer.

We use one of the shipped datamart exports for the example. This is a model very similar to one used in some of the ADM PowerPoint/Excel deep dive examples. You can change this notebook to apply to your own data.

[2]:

import polars as pl
import numpy as np

import plotly.express as px
from math import log
from great_tables import GT
from pdstools import datasets
from pdstools.utils import cdh_utils

[3]:

model_name = "AutoNew84Months"
predictor_name = "Customer.NetWealth"
channel = "Web"

For the example we pick one particular model over a channel. To explain the ADM model report, we use one of the active predictors as an example. Swap for any other predictor when using different data.

[4]:

dm = datasets.cdh_sample()

model = dm.combined_data.filter(
    (pl.col("Name") == model_name) & (pl.col("Channel") == channel)
)

modelpredictors = (
    dm.combined_data.join(
        model.select(pl.col("ModelID").unique()), on="ModelID", how="inner"
    )
    .filter(pl.col("EntryType") != "Inactive")
    .with_columns(
        Action=pl.concat_str(["Issue", "Group"], separator="/"),
        PredictorName=pl.col("PredictorName").cast(pl.Utf8),
    )
    .collect()
)

predictorbinning = modelpredictors.filter(
    pl.col("PredictorName") == predictor_name
).sort("BinIndex")

Model Overview

The selected model is shown below. Only the currently active predictors are used for the propensity calculation, so only showing those.

[6]:
Overview
Action Sales/AutoLoans
Channel Web
Name AutoNew84Months
Active Predictors Classifier, Customer.Age, Customer.AnnualIncome, Customer.BusinessSegment, Customer.CLV, Customer.CLV_VALUE, Customer.CreditScore, Customer.Date_of_Birth, Customer.Gender, Customer.MaritalStatus, Customer.NetWealth, Customer.NoOfDependents, Customer.Prefix, Customer.RelationshipStartDate, Customer.RiskCode, Customer.WinScore, Customer.pyCountry, IH.Email.Outbound.Accepted.pxLastGroupID, IH.Email.Outbound.Accepted.pxLastOutcomeTime.DaysSince, IH.Email.Outbound.Accepted.pyHistoricalOutcomeCount, IH.Email.Outbound.Churned.pyHistoricalOutcomeCount, IH.Email.Outbound.Loyal.pxLastOutcomeTime.DaysSince, IH.Email.Outbound.Rejected.pyHistoricalOutcomeCount, IH.SMS.Outbound.Accepted.pxLastGroupID, IH.SMS.Outbound.Accepted.pyHistoricalOutcomeCount, IH.SMS.Outbound.Churned.pxLastOutcomeTime.DaysSince, IH.SMS.Outbound.Loyal.pxLastOutcomeTime.DaysSince, IH.SMS.Outbound.Loyal.pyHistoricalOutcomeCount, IH.SMS.Outbound.Rejected.pxLastGroupID, IH.SMS.Outbound.Rejected.pyHistoricalOutcomeCount, IH.Web.Inbound.Accepted.pxLastGroupID, IH.Web.Inbound.Accepted.pyHistoricalOutcomeCount, IH.Web.Inbound.Loyal.pxLastGroupID, IH.Web.Inbound.Loyal.pyHistoricalOutcomeCount, IH.Web.Inbound.Rejected.pxLastGroupID, IH.Web.Inbound.Rejected.pyHistoricalOutcomeCount, Param.ExtGroupCreditcards
Model Performance (AUC) 77.4901

Binning of the selected Predictor

The Model Report in Prediction Studio for this model will have a predictor binning plot like below.

All numbers can be derived from just the number of positives and negatives in each bin that are stored in the ADM Data Mart. The next sections will show exactly how that is done.

Predictor information
Predictor Name Customer.NetWealth
# Responses 1636.0
# Bins 8
Predictor Performance(AUC) 72.2077 
[8]:
Binning statistics
Range/Symbol Responses (%) Positives Positives (%) Negatives Negatives (%) Propensity (%) ZRatio Lift
<11684.56 26.70% 13.0 6.30% 423.0 29.60% 2.98% −11.19 0.24
[11684.56, 13732.56> 12.30% 24.0 11.70% 178.0 12.40% 11.88% −0.33 0.94
[13732.56, 16845.52> 16.30% 17.0 8.30% 250.0 17.50% 6.37% −4.26 0.51
[16845.52, 19139.28> 14.10% 51.0 24.80% 179.0 12.50% 22.17% 3.91 1.76
[19139.28, 20286.16> 5.50% 7.0 3.40% 83.0 5.80% 7.78% −1.71 0.62
[20286.16, 22743.76> 13.60% 53.0 25.70% 169.0 11.80% 23.87% 4.40 1.90
[22743.76, 23890.64> 5.50% 13.0 6.30% 77.0 5.40% 14.44% 0.52 1.15
>=23890.64 6.10% 28.0 13.60% 71.0 5.00% 28.28% 3.51 2.25
Total 100.10% 206.0 100.10% 1430.0 100.00% 117.77% −5.16 9.35

Bin Statistics

Positive and Negative ratios

Internally, ADM only keeps track of the total counts of positive and negative responses in each bin. Everything else is derived from those numbers. The percentages and totals are trivially derived, and the propensity is just the number of positives divided by the total. The numbers calculated here match the numbers from the datamart table exactly.

[9]:

binning_derived = predictorbinning.select(
    pl.col("BinSymbol").alias("Range/Symbol"),
    BinPositives.alias("Positives"),
    BinNegatives.alias("Negatives"),
    ((BinPositives + BinNegatives) / (sumPositives + sumNegatives)).alias(
        "Responses %"
    ),
    (BinPositives / sumPositives).alias("Positives %"),
    (BinNegatives / sumNegatives).alias("Negatives %"),
    (BinPositives / (BinPositives + BinNegatives)).round(4).alias("Propensity"),
)

pcts = ["Responses %", "Positives %", "Negatives %", "Propensity"]
GT(binning_derived).tab_header("Derived binning statistics").tab_style(
    style=style.text(weight="bold"), locations=loc.body(columns="Range/Symbol")
).tab_style(
    style=style.text(color="blue"),
    locations=loc.body(columns=pcts),
).fmt_percent(pcts).tab_options(table_margin_left=0)

[9]:
Derived binning statistics
Range/Symbol Positives Negatives Responses % Positives % Negatives % Propensity
<11684.56 13.0 423.0 26.65% 6.31% 29.58% 2.98%
[11684.56, 13732.56> 24.0 178.0 12.35% 11.65% 12.45% 11.88%
[13732.56, 16845.52> 17.0 250.0 16.32% 8.25% 17.48% 6.37%
[16845.52, 19139.28> 51.0 179.0 14.06% 24.76% 12.52% 22.17%
[19139.28, 20286.16> 7.0 83.0 5.50% 3.40% 5.80% 7.78%
[20286.16, 22743.76> 53.0 169.0 13.57% 25.73% 11.82% 23.87%
[22743.76, 23890.64> 13.0 77.0 5.50% 6.31% 5.38% 14.44%
>=23890.64 28.0 71.0 6.05% 13.59% 4.97% 28.28%

Lift

Lift is the ratio of the propensity in a particular bin over the average propensity. So a value of 1 is the average, larger than 1 means higher propensity, smaller means lower propensity:

[10]:

positives = pl.col("Positives")
negatives = pl.col("Negatives")
sumPositives = pl.sum("Positives")
sumNegatives = pl.sum("Negatives")
GT(
    binning_derived.select(
        "Range/Symbol",
        "Positives",
        "Negatives",
        (
            (positives / (positives + negatives))
            / (sumPositives / (positives + negatives).sum())
        )
        .round(4)
        .alias("Lift"),
    )
).tab_style(
    style=style.text(weight="bold"), locations=loc.body(columns="Range/Symbol")
).tab_style(
    style=style.text(color="blue"), locations=loc.body(columns=["Lift"])
).tab_options(table_margin_left=0)

[10]:
Range/Symbol Positives Negatives Lift
<11684.56 13.0 423.0 0.2368
[11684.56, 13732.56> 24.0 178.0 0.9436
[13732.56, 16845.52> 17.0 250.0 0.5057
[16845.52, 19139.28> 51.0 179.0 1.761
[19139.28, 20286.16> 7.0 83.0 0.6177
[20286.16, 22743.76> 53.0 169.0 1.896
[22743.76, 23890.64> 13.0 77.0 1.1471
>=23890.64 28.0 71.0 2.2462

Z-Ratio

The Z-Ratio is also a measure of the how the propensity in a bin differs from the average, but takes into account the size of the bin and thus is statistically more relevant. It represents the number of standard deviations from the average, so centers around 0. The wider the spread, the better the predictor is.

\[\frac{posFraction-negFraction}{\sqrt(\frac{posFraction*(1-posFraction)}{\sum positives}+\frac{negFraction*(1-negFraction)}{\sum negatives})}\]

See the calculation here, which is also included in cdh_utils’ zRatio().

[11]:

def z_ratio(
    pos_col: pl.Expr = pl.col("BinPositives"), neg_col: pl.Expr = pl.col("BinNegatives")
) -> pl.Expr:
    def get_fracs(pos_col=pl.col("BinPositives"), neg_col=pl.col("BinNegatives")):
        return pos_col / pos_col.sum(), neg_col / neg_col.sum()

    def z_ratio_impl(
        pos_fraction_col=pl.col("posFraction"),
        neg_fraction_col=pl.col("negFraction"),
        positives_col=pl.sum("BinPositives"),
        negatives_col=pl.sum("BinNegatives"),
    ):
        return (
            (pos_fraction_col - neg_fraction_col)
            / (
                (pos_fraction_col * (1 - pos_fraction_col) / positives_col)
                + (neg_fraction_col * (1 - neg_fraction_col) / negatives_col)
            ).sqrt()
        ).alias("ZRatio")

    return z_ratio_impl(*get_fracs(pos_col, neg_col), pos_col.sum(), neg_col.sum())


GT(
    binning_derived.select(
        "Range/Symbol", "Positives", "Negatives", "Positives %", "Negatives %"
    ).with_columns(z_ratio(positives, negatives).round(4))
).tab_style(
    style=style.text(weight="bold"), locations=loc.body(columns="Range/Symbol")
).tab_style(
    style=style.text(color="blue"), locations=loc.body(columns=["ZRatio"])
).fmt_percent(pl.selectors.ends_with("%")).tab_options(table_margin_left=0)

[11]:
Range/Symbol Positives Negatives Positives % Negatives % ZRatio
<11684.56 13.0 423.0 6.31% 29.58% -11.1869
[11684.56, 13732.56> 24.0 178.0 11.65% 12.45% -0.3321
[13732.56, 16845.52> 17.0 250.0 8.25% 17.48% -4.2647
[16845.52, 19139.28> 51.0 179.0 24.76% 12.52% 3.9082
[19139.28, 20286.16> 7.0 83.0 3.40% 5.80% -1.7118
[20286.16, 22743.76> 53.0 169.0 25.73% 11.82% 4.3976
[22743.76, 23890.64> 13.0 77.0 6.31% 5.38% 0.5156
>=23890.64 28.0 71.0 13.59% 4.97% 3.5129

Predictor AUC

The predictor AUC is the univariate performance of this predictor against the outcome. This too can be derived from the positives and negatives and there is a convenient function in pdstools to calculate it directly from the positives and negatives.

This function is implemented in cdh_utils: cdh_utils.auc_from_bincounts().

[12]:

pos = binning_derived.get_column("Positives")
neg = binning_derived.get_column("Negatives")
probs = binning_derived.get_column("Propensity")
order = probs.arg_sort()
FPR = pl.Series([0.0]).extend(neg[order].cum_sum() / neg[order].sum())
TPR = pl.Series([0.0]).extend(pos[order].cum_sum() / pos[order].sum())
if TPR[1] < 1 - FPR[1]:
    FPR, TPR = TPR, FPR

---------------------------------------------------------------------------
SchemaError                               Traceback (most recent call last)
Cell In[12], line 5
      3 probs = binning_derived.get_column("Propensity")
      4 order = probs.arg_sort()
----> 5 FPR = pl.Series([0.0]).extend(neg[order].cum_sum() / neg[order].sum())
      6 TPR = pl.Series([0.0]).extend(pos[order].cum_sum() / pos[order].sum())
      7 if TPR[1] < 1 - FPR[1]:

File ~/work/pega-datascientist-tools/pega-datascientist-tools/.venv/lib/python3.11/site-packages/polars/series/series.py:3005, in Series.extend(self, other)
   2951 """
   2952 Extend the memory backed by this Series with the values from another.
   2953
   (...)
   3002 1
   3003 """
   3004 try:
-> 3005     self._s.extend(other._s)
   3006 except RuntimeError as exc:
   3007     if str(exc) == "Already mutably borrowed":

SchemaError: type Float32 is incompatible with expected type Float64

[13]:

pos = binning_derived.get_column("Positives").to_numpy()
neg = binning_derived.get_column("Negatives").to_numpy()
probs = binning_derived.get_column("Propensity").to_numpy()
order = np.argsort(probs)

FPR = np.cumsum(neg[order]) / np.sum(neg[order])
TPR = np.cumsum(pos[order]) / np.sum(pos[order])
TPR = np.insert(TPR, 0, 0, axis=0)
FPR = np.insert(FPR, 0, 0, axis=0)
# Checking whether classifier labels are correct
if TPR[1] < 1 - FPR[1]:
    temp = FPR
    FPR = TPR
    TPR = temp
auc = cdh_utils.auc_from_bincounts(pos=pos, neg=neg, probs=probs)

fig = px.line(
    x=[1 - x for x in FPR],
    y=TPR,
    labels=dict(x="Specificity", y="Sensitivity"),
    title=f"AUC = {auc.round(3)}",
    width=700,
    height=700,
    range_x=[1, 0],
    template="none",
)
fig.add_shape(type="line", line=dict(dash="dash"), x0=1, x1=0, y0=0, y1=1)
fig.show()

Naive Bayes and Log Odds

The basis for the Naive Bayes algorithm is Bayes’ Theorem:

\[p(C_k|x) = \frac{p(x|C_k)*p(C_k)}{p(x)}\]

with \(C_k\) the outcome and \(x\) the customer. Bayes’ theorem turns the question “what’s the probability to accept this action given a customer” around to “what’s the probability of this customer given an action”. With the independence assumption, and after applying a log odds transformation we get a log odds score that can be calculated efficiently and in a numerically stable manner:

\[log\ odds\ score = \sum_{p\ \in\ active\ predictors}log(p(x_p|Positive)) + log(p_{positive}) - \sum_plog(p(x_p|Negative)) - log(p_{negative})\]

note that the prior can be written as:

\[log(p_{positive}) - log(p_{negative}) = log(\frac{TotalPositives}{Total})-log(\frac{TotalNegatives}{Total}) = log(TotalPositives) - log(TotalNegatives)\]

Predictor Contribution

The contribution (conditional log odds) of an active predictor \(p\) for bin \(i\) with the number of positive and negative responses in \(Positives_i\) and \(Negatives_i\) is calculated as (note the “laplace smoothing” to avoid log 0 issues):

\[contribution_p = \log(Positives_i+\frac{1}{nBins}) - \log(Negatives_i+\frac{1}{nBins}) - \log(1+\sum_{i\ = 1..nBins}{Positives_i}) + \log(1+\sum_i{Negatives_i})\]
[14]:

N = binning_derived.shape[0]
GT(
    binning_derived.with_columns(
        LogOdds=(pl.col("Positives %") / pl.col("Negatives %")).log().round(5),
        ModifiedLogOdds=(
            ((positives + 1 / N).log() - (positives.sum() + 1).log())
            - ((negatives + 1 / N).log() - (negatives.sum() + 1).log())
        ).round(5),
    ).drop("Responses %", "Propensity")
).tab_style(
    style=style.text(weight="bold"), locations=loc.body(columns="Range/Symbol")
).tab_style(
    style=style.text(color="blue"),
    locations=loc.body(columns=["LogOdds", "ModifiedLogOdds"]),
).fmt_percent(pl.selectors.ends_with("%")).tab_options(table_margin_left=0)

[14]:
Range/Symbol Positives Negatives Positives % Negatives % LogOdds ModifiedLogOdds
<11684.56 13.0 423.0 6.31% 29.58% -1.54487 -1.53974
[11684.56, 13732.56> 24.0 178.0 11.65% 12.45% -0.06618 -0.06583
[13732.56, 16845.52> 17.0 250.0 8.25% 17.48% -0.75069 -0.74801
[16845.52, 19139.28> 51.0 179.0 24.76% 12.52% 0.68199 0.6796
[19139.28, 20286.16> 7.0 83.0 3.40% 5.80% -0.53538 -0.52333
[20286.16, 22743.76> 53.0 169.0 25.73% 11.82% 0.77795 0.77542
[22743.76, 23890.64> 13.0 77.0 6.31% 5.38% 0.1587 0.1625
>=23890.64 28.0 71.0 13.59% 4.97% 1.00708 1.00563

Propensity mapping

Log odds contribution for all the predictors

The final score is loosely referred to as “the average contribution” but in fact is a little more nuanced. The final score is calculated as:

\[score = \frac{\log(1 + TotalPositives) – \log(1 + TotalNegatives) + \sum_p contribution_p}{1 + nActivePredictors}\]

Here, \(TotalPositives\) and \(TotalNegatives\) are the total number of positive and negative responses to the model.

Below an example. From all the active predictors of the model we pick a value (in the middle for numerics, first symbol for symbolics) and show the (modified) log odds. The final score is calculated per the above formula, and this is the value that is mapped to a propensity value by the classifier (which is constructed using the PAV(A) algorithm).

[15]:
PredictorName Value Bin Positives Negatives LogOdds
Customer.Age 34.56 4 9.0 198.0 -1.1459
Customer.AnnualIncome -24043.049 1 74.0 1166.0 -0.8197
Customer.BusinessSegment middleSegmentPlus 1 96.0 970.0 -0.3764
Customer.CLV NON-MISSING 1 111.0 570.0 0.3009
Customer.CLV_VALUE 1345.52 4 31.0 297.0 -0.3227
Customer.CreditScore 518.92 3 33.0 205.0 0.1105
Customer.Date_of_Birth 18773.504 5 28.0 152.0 0.2446
Customer.Gender U 1 52.0 481.0 -0.2855
Customer.MaritalStatus No Resp+ 1 67.0 745.0 -0.4708
Customer.NetWealth 17992.398 4 51.0 179.0 0.6796
Customer.NoOfDependents 0.0 1 111.0 850.0 -0.0997
Customer.Prefix Mrs. 1 64.0 552.0 -0.2167
Customer.RelationshipStartDate 1426.4596 4 16.0 117.0 -0.0502
Customer.RiskCode R4 1 36.0 329.0 -0.2709
Customer.WinScore 66.600006 4 39.0 102.0 0.9738
Customer.pyCountry USA 1 99.0 776.0 -0.1227
IH.Email.Outbound.Accepted.pxLastGroupID HomeLoans 3 25.0 218.0 -0.2272
IH.Email.Outbound.Accepted.pxLastOutcomeTime.DaysSince -55.88436 2 145.0 881.0 0.1305
IH.Email.Outbound.Accepted.pyHistoricalOutcomeCount 1.5 2 30.0 351.0 -0.5201
IH.Email.Outbound.Churned.pyHistoricalOutcomeCount None 1 143.0 898.0 0.1015
IH.Email.Outbound.Loyal.pxLastOutcomeTime.DaysSince None 1 129.0 1071.0 -0.1788
IH.Email.Outbound.Rejected.pyHistoricalOutcomeCount 83.16 3 24.0 218.0 -0.2678
IH.SMS.Outbound.Accepted.pxLastGroupID Account 4 45.0 316.0 -0.0133
IH.SMS.Outbound.Accepted.pyHistoricalOutcomeCount 9.02 4 6.0 96.0 -0.822
IH.SMS.Outbound.Churned.pxLastOutcomeTime.DaysSince -20.5537 2 9.0 27.0 0.8492
IH.SMS.Outbound.Loyal.pxLastOutcomeTime.DaysSince None 1 165.0 1240.0 -0.0797
IH.SMS.Outbound.Loyal.pyHistoricalOutcomeCount None 1 165.0 1240.0 -0.0797
IH.SMS.Outbound.Rejected.pxLastGroupID Account 2 47.0 357.0 -0.0905
IH.SMS.Outbound.Rejected.pyHistoricalOutcomeCount 102.72 4 12.0 117.0 -0.3356
IH.Web.Inbound.Accepted.pxLastGroupID DepositAccounts 3 53.0 397.0 -0.0779
IH.Web.Inbound.Accepted.pyHistoricalOutcomeCount 11.04 5 25.0 164.0 0.0558
IH.Web.Inbound.Loyal.pxLastGroupID MISSING 1 100.0 857.0 -0.2119
IH.Web.Inbound.Loyal.pyHistoricalOutcomeCount 4.52 3 30.0 212.0 -0.0172
IH.Web.Inbound.Rejected.pxLastGroupID Account 2 81.0 546.0 0.0279
IH.Web.Inbound.Rejected.pyHistoricalOutcomeCount 111.08 4 35.0 306.0 -0.2317
Param.ExtGroupCreditcards NON-MISSING 1 136.0 721.0 0.2684
Final Score None None None None -0.14933270233219137

Classifier

The success rate is defined as \(\frac{positives}{positives+negatives}\) per bin.

The adjusted propensity that is returned is a small modification (Laplace smoothing) to this and calculated as \(\frac{0.5+positives}{1+positives+negatives}\) so empty models return a propensity of 0.5.

[16]:
Index Bin Positives Negatives Cum. Total (%) Propensity (%) Adjusted Propensity (%) Cum Positives (%) ZRatio Lift(%)
1 <-0.21 17.0 443.0 28.12% 3.70% 3.80% 8.25% -9.9945 1.96%
2 [-0.21, -0.185> 8.0 133.0 36.74% 5.67% 5.99% 12.14% -3.4954 3.00%
3 [-0.185, -0.175> 3.0 48.0 39.85% 5.88% 6.73% 13.59% -1.9775 3.11%
4 [-0.175, -0.105> 28.0 370.0 64.18% 7.04% 7.14% 27.18% -4.6281 3.72%
5 [-0.105, -0.095> 4.0 51.0 67.54% 7.27% 8.04% 29.13% -1.5054 3.85%
6 [-0.095, -0.09> 2.0 19.0 68.83% 9.52% 11.36% 30.10% -0.4788 5.04%
7 [-0.09, -0.065> 9.0 77.0 74.08% 10.47% 10.92% 34.47% -0.6578 5.54%
8 [-0.065, -0.02> 30.0 154.0 85.33% 16.30% 16.49% 49.03% 1.4644 8.63%
9 [-0.02, 0.03> 37.0 65.0 91.56% 36.27% 36.41% 66.99% 4.913 19.21%
10 [0.03, 0.06> 20.0 29.0 94.56% 40.82% 41.00% 76.70% 3.664 21.61%
11 [0.06, 0.12> 30.0 33.0 98.41% 47.62% 47.66% 91.26% 4.9229 25.21%
12 [0.12, 0.125> 2.0 2.0 98.66% 50.00% 50.00% 92.23% 1.2039 26.47%
13 [0.125, 0.13> 4.0 2.0 99.02% 66.67% 64.29% 94.17% 1.8644 35.30%
14 [0.13, 0.995> 8.0 3.0 99.69% 72.73% 70.83% 98.06% 2.7182 38.51%
15 >=0.995 4.0 1.0 100.00% 80.00% 75.00% 100.00% 1.9418 42.36%

Final Propensity

Below the classifier mapping. On the x-axis the binned scores (log odds values), on the y-axis the Propensity. Note the returned propensities are following a slightly adjusted formula, see the table above. The bin that contains the calculated final score is highlighted.

[17]:

score = propensity_mapping.filter(PredictorName="Final Score")["LogOdds"][0]
score_bin = (
    modelpredictors.filter(pl.col("EntryType") == "Classifier")
    .select(
        pl.col("BinSymbol").filter(
            pl.lit(score).is_between(pl.col("BinLowerBound"), pl.col("BinUpperBound"))
        )
    )
    .item()
)
score_responses = modelpredictors.filter(
    (pl.col("EntryType") == "Classifier") & (pl.col("BinSymbol") == score_bin)
)["BinResponseCount"][0]

score_bin_index = (
    modelpredictors.filter(pl.col("EntryType") == "Classifier")["BinSymbol"]
    .to_list()
    .index(score_bin)
)

score_propensity = classifier.row(score_bin_index, named=True)[
    "Adjusted Propensity (%)"
]

adjusted_propensity = (
    modelpredictors.filter(pl.col("EntryType") == "Classifier")
    .with_columns(
        AdjustedPropensity=(
            (0.5 + BinPositives) / (1 + BinPositives + BinNegatives)
        ).round(5),
    )
    .select(
        pl.col("AdjustedPropensity").filter(
            (pl.col("BinLowerBound") < score) & (pl.col("BinUpperBound") > score)
        )
    )["AdjustedPropensity"][0]
)
fig = dm.plot.score_distribution(
    model_id=modelpredictors.get_column("ModelID").unique().to_list()[0]
).add_annotation(
    x=score_bin,
    y=score_propensity / 100,
    text=f"Returned propensity: {score_propensity*100:.2f}%",
    bgcolor="#FFFFFF",
    bordercolor="#000000",
    showarrow=False,
    yref="y2",
    opacity=0.7,
)
bin_index = list(fig.data[0]["x"]).index(score_bin)
fig.data[0]["marker_color"] = (
    ["grey"] * bin_index
    + ["#1f77b4"]
    + ["grey"] * (classifier.shape[0] - bin_index - 1)
)
fig

[ ]: