Register a Model Explainer

TL;DR

A Model Explainer measures how much each input contributes to a model’s prediction. Enable it on the Model’s Formula section, choose Feature Importance (a contribution per input) or Reason Code (adverse-action codes), and write the explainer definition. Once enabled, every simulation writes an explainer_<model_alias> column and the Feature Importance report is generated automatically.

What is it?

A Model Explainer is logic attached to a registered Model that measures the contribution of each input to a prediction, either for a single record (a local contribution) or across the whole sample (feature importance). It answers “why did the model produce this score?”, which is essential for model risk management, adverse-action notices, and debugging.

The explainer is defined while registering the Model, in the Formula section. This page walks through that logic in detail.

Two explainer usages

You pick one Explainer Usage when you enable the explainer. It determines both what your definition returns and how the values are used downstream.

Explainer Usage	What your definition returns	Use for
Feature Importance	A dictionary keyed by input alias, mapping each input to its contribution (for example, a SHAP value).	Showing which inputs drove a score; drives the Feature Importance report.
Reason Code	A list of registered reason codes for the prediction.	Adverse-action / decline reasons, surfaced in a Policy via a model-driven reason code.

How to Register

Follow these steps in the Formula section of the Model form.

Enable the Model Explainer

In the Model’s Formula section, below the definition editor, toggle Model Explainer on.

Choose the Explainer Usage

Set Explainer Usage to Feature Importance (a contribution per input) or Reason Code (a list of reason codes). The expected return value of your definition depends on this choice.

Write the Explainer Definition

Inside the Explainer Definition editor you have access to:

• Every model input, by its alias (the same aliases from the model’s Input section).
• A model variable. What model is depends on the Model Input Type:
  • Pickle / PMML / Custom - the initialized model object. Call its native methods (for example, model.predict_proba(...)) or hand it to a library like SHAP.
  • Python / H2O MOJO / ONNX / Lookup - the model function defined in the model definition. Call it by passing the inputs, model(input1, input2, ...).
• For Reason Code usage only - each registered Reason Code you selected, available as a variable by its alias.

Write your logic and return the value that matches your Explainer Usage:

Pickle / PMML / Custom

Feature Importance usage. For Pickle / PMML / Custom, model is the initialized model object - call its native methods (for example, model.predict_proba(...)) or hand it to a library like SHAP. Return a dictionary keyed by input alias, mapping each input to its contribution.

Example 1 - SHAP values:

import shap

feature_names = ['income_de', 'fico_de']
explainer = shap.TreeExplainer(model=model, feature_names=feature_names)
shap_values = explainer.shap_values([[income_de, fico_de]])[0]

return {feat: float(val) for feat, val in zip(feature_names, shap_values)}

Example 2 - model.predict_proba (score, then re-score with each input zeroed out):

feature_names = ['income_de', 'fico_de']
row = [income_de, fico_de]

base = model.predict_proba([row])[0][1]

contributions = {}
for i, feat in enumerate(feature_names):
    perturbed = row.copy()
    perturbed[i] = 0
    contributions[feat] = float(base - model.predict_proba([perturbed])[0][1])

return contributions

Python / H2O MOJO / ONNX / Lookup

Feature Importance usage. For Python / H2O MOJO / ONNX / Lookup, model is the model function from the model definition - call it by passing the inputs, model(input1, input2, ...). Return a dictionary of contributions keyed by input alias.

Example 1 - derive a contribution from each input directly (toy example - swap in your own attribution logic):

return {
    'annual_income':        round(annual_income / 100000, 4),
    'debt_to_income_ratio': round(-debt_to_income_ratio * 0.35, 4),
}

Example 2 - call the model, then re-call it with each input zeroed out:

base = model(income_de, fico_de)

contributions = {
    'income_de': float(base - model(0, fico_de)),
    'fico_de':   float(base - model(income_de, 0)),
}

return contributions

Reason Code

Reason Code usage. Each Reason Code you selected is available as a variable by its alias. Return the list of codes that apply to the prediction:

# high_dti and low_fico are registered Reason Codes, available by alias
reasons = []
if debt_to_income_ratio > 0.43:
    reasons.append(high_dti)
if fico < 620:
    reasons.append(low_fico)

return reasons

Save the Model

Save the Model. The explainer is versioned with it and runs on every subsequent simulation.

Where the explainer values land

When the explainer is enabled, every model simulation writes an explainer_<model_alias> column into the simulation output (named after the model’s output alias), alongside the inputs, the dependent variable, and the model output. For Feature Importance, each cell holds the dictionary of per-input contributions for that record; for Reason Code, it holds the list of reason codes returned (represented by each Reason Code’s registered Code).

The output file is Parquet by default. You can open it in the Notebook and convert the explainer_<model_alias> column into a Pandas DataFrame for further analysis.

The Feature Importance report

The Feature Importance report ranks each input by its aggregate contribution across the sample (the individual contributions from the explainer, summed over every record). It is a standard report, so it runs automatically as part of a model simulation. It only produces output when the Model has an explainer and Explainer Usage is Feature Importance; with Reason Code (or no explainer) it produces nothing. You do not select it manually.

What’s Next

• Run a Simulation to generate the explainer column and the Feature Importance report.
• Send the Model for approval once the explainer output looks right.
• Use reason codes in a Policy Decision Workflow via a model-driven reason code.