Model Results Phase 3 ✅ APPROVED

XGBoost + LightGBM trained on ≤2023; validated on 2024–2025. Binary target: fatal+serious vs other. AUC values below update from live JSON.

Model Comparison 4 ratified models · ATHENA ratified Phase 3 APPROVED

Model	Role	AUC-ROC	AUC-PR	RMSE	MAE-ord	Brier	ECE
XGBoost / LightGBM	Primary	0.624 / 0.625	0.777 / 0.779	—	—	0.239 / 0.238	0.163 / 0.166
Ordered Logit	Primary	—	—	—	—	—	—
Random Forest	Baseline	—	—	—	—	—	—
Negative Binomial II	Supplementary	—	—	—	—	—	—

XGBoost + LightGBM are primary models. Ordered Logit, NB2, Random Forest pending. Temporal split: train ≤2023 | test 2024–2025. Bias toward recall for safety applications (threshold explorer below).

Speed–Severity Gradient Key finding: lower speeds = higher serious/fatal rate Inverted Relationship

Standard intuition: high speed = high risk. Victoria data: serious/fatal rate is 27% higher at 30km/h vs 100km/h. This is not a model error — it reflects urban road reality: more pedestrians, cyclists, conflict points at low speeds. Safety interventions should target urban low-speed zones (30–60km/h), not just high-speed arterials.

Threshold Explorer Precision–Recall tradeoff Safety: bias recall

Recall ↑Threshold: 0.50Precision ↑

Feature Importance Top 10

Speed limit

—

Time of day

—

Road curvature

—

Weather

—

Lighting

—

Surface type

—

Alcohol involvement

—

DCA code

—

Risk Tier Breakdown % of dataset

Tier	Count	Pct	Colour

Confusion Matrix