Information Value Analysis
Purpose
The information value analysis identifies which impact multipliers (impact differentiators) have the most consequential uncertainty for portfolio optimization decisions. By quantifying how much each multiplier's uncertainty affects funding allocation choices, this analysis helps prioritize which parameters would benefit most from additional research or data collection to reduce uncertainty.
Overview
The analysis uses multiple statistical methods to measure the relationship between each impact multiplier and funding decisions across all simulations. Each method provides a different perspective on how influential a multiplier is in determining optimal portfolio choices.
Methods
The analysis uses 4 statistical methods by default to measure the relationship between each impact multiplier and funding decisions:
1. Mean Absolute Spearman Correlation
Description: For every multiplier, compute the Spearman correlation between that multiplier and each organisation's simulated funding. Take the absolute value and average these correlations across organisations to obtain a single 0-to-1 score.
Interpretation: A larger score means the multiplier is more strongly and consistently aligned—positively or negatively—with funding decisions; 1 denotes a perfect monotonic link, 0 no monotonic link.
Use case: Best for identifying linear and monotonic relationships between multipliers and funding decisions.
2. Mutual Information
Description: Discretise multiplier and funding values, then calculate mutual information between them for each organisation. Average the mutual-information scores to capture all—linear and non-linear—dependencies in a single measure (expressed in bits).
Interpretation: Higher values indicate that knowing this multiplier meaningfully reduces uncertainty about funding decisions, regardless of the functional form of the relationship.
Use case: Captures both linear and non-linear relationships, making it more comprehensive than correlation analysis.
Requirements: Requires the infotheo package to be installed.
3. Expected Value Regret
Description: For each multiplier, trim the top 25% (high side) or bottom 25% (low side) of its simulated range, recompute expected values for all portfolios, and calculate the extra value that could be captured by switching within the top portfolio(s) under this reduced-uncertainty view.
Interpretation: Shows separate values for reducing uncertainty on the 'high' vs 'low' side by 25% for the top portfolio(s), helping identify which direction of uncertainty reduction is most valuable.
Use case: Directly measures the economic value of reducing uncertainty in each multiplier, providing actionable insights for research prioritization.
Parameters:
- reduce_uncertainty_by: Fraction of uncertainty to reduce (default: 0.25)
- top_n_portfolio: Number of top portfolios to consider (default: 1)
4. Jensen-Shannon Divergence
Description: Slice the multiplier into 10 equal-frequency bins, compute the average normalised funding profile in each bin, then average the Jensen–Shannon divergence between every pair of adjacent bins.
Interpretation: A larger divergence means the multiplier triggers sharper shifts in funding priorities between neighbouring value ranges; values near 0 imply little change.
Use case: Measures how much funding patterns change as multiplier values increase, capturing threshold effects and non-linear impacts.
Requirements: Skips multipliers with fewer than 5 unique values.