--- title: "Register a Model Explainer" description: "Register a Model Explainer to surface feature contributions behind a model's predictions." --- import { Tabs, TabItem } from '@astrojs/starlight/components';
TL;DR
A Model Explainer measures how much each input contributes to a model's prediction. Enable it on the Model's Formula section, choose Feature Importance (a contribution per input) or Reason Code (adverse-action codes), and write the explainer definition. Once enabled, every simulation writes an explainer_<model_alias> column and the Feature Importance report is generated automatically.
## What is it? A **Model Explainer** is logic attached to a registered [Model](/user-guide/how-tos/register-a-model-definition/) that measures the contribution of each input to a prediction, either for a single record (a local contribution) or across the whole sample (feature importance). It answers "why did the model produce this score?", which is essential for model risk management, adverse-action notices, and debugging. The explainer is defined **while registering the Model**, in the **Formula** section. This page walks through that logic in detail. ## Two explainer usages You pick one **Explainer Usage** when you enable the explainer. It determines both what your definition returns and how the values are used downstream.
Explainer Usage What your definition returns Use for
Feature Importance A dictionary keyed by input alias, mapping each input to its contribution (for example, a SHAP value). Showing which inputs drove a score; drives the Feature Importance report.
Reason Code A list of registered reason codes for the prediction. Adverse-action / decline reasons, surfaced in a Policy via a model-driven reason code.
## How to Register Follow these steps in the **Formula** section of the [Model](/user-guide/how-tos/register-a-model-definition/) form.
#### Enable the Model Explainer In the Model's **Formula** section, below the definition editor, toggle **Model Explainer** on.
#### Choose the Explainer Usage Set **Explainer Usage** to `Feature Importance` (a contribution per input) or `Reason Code` (a list of reason codes). The expected return value of your definition depends on this choice.
#### Write the Explainer Definition Inside the **Explainer Definition** editor you have access to: - **Every model input**, by its alias (the same aliases from the model's **Input** section). - **A `model` variable.** What `model` is depends on the **Model Input Type**: - **Pickle / PMML / Custom** - the **initialized model object**. Call its native methods (for example, `model.predict_proba(...)`) or hand it to a library like SHAP. - **Python / H2O MOJO / ONNX / Lookup** - the **model function** defined in the model definition. Call it by passing the inputs, `model(input1, input2, ...)`. - **For Reason Code usage only** - each registered Reason Code you selected, available as a variable by its alias. Write your logic and `return` the value that matches your Explainer Usage: `Feature Importance` usage. For **Pickle / PMML / Custom**, `model` is the **initialized model object** - call its native methods (for example, `model.predict_proba(...)`) or hand it to a library like SHAP. Return a dictionary keyed by input alias, mapping each input to its contribution. **Example 1 - SHAP values:** ```python import shap feature_names = ['income_de', 'fico_de'] explainer = shap.TreeExplainer(model=model, feature_names=feature_names) shap_values = explainer.shap_values([[income_de, fico_de]])[0] return {feat: float(val) for feat, val in zip(feature_names, shap_values)} ``` **Example 2 - `model.predict_proba` (score, then re-score with each input zeroed out):** ```python feature_names = ['income_de', 'fico_de'] row = [income_de, fico_de] base = model.predict_proba([row])[0][1] contributions = {} for i, feat in enumerate(feature_names): perturbed = row.copy() perturbed[i] = 0 contributions[feat] = float(base - model.predict_proba([perturbed])[0][1]) return contributions ``` `Feature Importance` usage. For **Python / H2O MOJO / ONNX / Lookup**, `model` is the **model function** from the model definition - call it by passing the inputs, `model(input1, input2, ...)`. Return a dictionary of contributions keyed by input alias. **Example 1 - derive a contribution from each input directly** (toy example - swap in your own attribution logic): ```python return { 'annual_income': round(annual_income / 100000, 4), 'debt_to_income_ratio': round(-debt_to_income_ratio * 0.35, 4), } ``` **Example 2 - call the model, then re-call it with each input zeroed out:** ```python base = model(income_de, fico_de) contributions = { 'income_de': float(base - model(0, fico_de)), 'fico_de': float(base - model(income_de, 0)), } return contributions ``` `Reason Code` usage. Each Reason Code you selected is available as a variable by its alias. Return the **list** of codes that apply to the prediction: ```python # high_dti and low_fico are registered Reason Codes, available by alias reasons = [] if debt_to_income_ratio > 0.43: reasons.append(high_dti) if fico < 620: reasons.append(low_fico) return reasons ```
#### Save the Model Save the Model. The explainer is versioned with it and runs on every subsequent simulation.
## Where the explainer values land When the explainer is enabled, every model simulation writes an `explainer_` column into the simulation output (named after the model's output alias), alongside the inputs, the dependent variable, and the model output. For `Feature Importance`, each cell holds the dictionary of per-input contributions for that record; for `Reason Code`, it holds the list of reason codes returned (represented by each Reason Code's registered Code). The output file is Parquet by default. You can open it in the [Notebook](/user-guide/how-tos/use-the-notebook/) and convert the `explainer_` column into a Pandas DataFrame for further analysis. ## The Feature Importance report The **Feature Importance** report ranks each input by its aggregate contribution across the sample (the individual contributions from the explainer, summed over every record). It is a **standard report**, so it runs automatically as part of a model simulation. It only produces output when the Model has an explainer **and** Explainer Usage is `Feature Importance`; with `Reason Code` (or no explainer) it produces nothing. You do not select it manually. ## What's Next - [Run a Simulation](/user-guide/how-tos/run-a-simulation/) to generate the explainer column and the Feature Importance report. - Send the Model for [approval](/user-guide/how-tos/approve-an-object/) once the explainer output looks right. - Use reason codes in a [Policy Decision Workflow](/user-guide/how-tos/register-policy-decision-workflow/) via a model-driven reason code.