Overview
Two analyses, each computed with two feature-selection methods. AUC uses pROC direction="<" throughout.
Key finding: supervised correlation feature-selection overfits — it transfers worse across cohorts and gives lower combined CV-AUC than unsupervised MAD selection.
Transfer model summary
Each source cohort's whole-dataset GLM (top-N features): number of features / non-zero coefficients, honest within-cohort CV (Source_CV_AUC), and transfer AUC applied to each other cohort.
Source_CV_AUC = mean per-fold AUC of within-cohort CV (fold-internal feature selection). AUC_<cohort> = model applied to that other cohort; the source's own column is blank.
Transfer AUC heatmap — one cohort → another
Rows = source→target, columns = feature cutoff. Cells confined to a square grid.
Diagonal (source=target) is omitted — see the summary table above for each source's own-cohort CV.
Feature overlap across the 3 cohort models (Venn)
The top-N selected features of each cohort's transfer model at a given cutoff, and how much they overlap. Robust biomarkers sit in the centre (all 3).
counts (miRNAs)
percent of union
Set sizes equal the cutoff (top-N by MAD or |corr|); overlaps show shared miRNAs. All-3 (centre) miRNAs are the cross-cohort consensus. Percentages are each region as a share of the union of all three selected sets (sums to 100%).
Consensus feature importance
For the miRNAs shared by all three cohort models at a cutoff, the |coefficient| in each cohort's GLM (one bar group per feature, ranked by mean |coef|).
Empty = no miRNA is shared by all three cohort models at that cutoff (common at very small N). Try v20/v50.
Combined-cohort AUC — 4 batch-handling variants
Pooled 3-cohort model, mean per-fold AUC ± 95% CI vs feature cutoff. Compare batch variants, learners, and the two FS methods.
All four pool the same 300 + 166 + 82 samples and differ only in how cohort/batch is handled. Hover a card for the technical detail.
ROC curves — own-cohort
Each cohort's within-cohort 10-fold CV (fold-internal feature selection, pooled out-of-fold predictions, direction="<"). Pick a dataset × FS method; each plot overlays the ROC at every top-var cutoff.
ROC curves — transfer (source → target)
A GLM refit on the whole source cohort applied to another cohort; this is that transfer's actual prediction ROC (direction="<"). Each plot overlays every top-var cutoff; curves below the diagonal (AUC<0.5) are anti-predictive transfers.
Operating point & optimal cutoff
Pick one model, then slide the probability cutoff (predict recurrence when prob ≥ cutoff) to read sensitivity, specificity and predictive values at that threshold. The red star marks the Youden-optimal cutoff (maximizes sensitivity + specificity − 1) on the 0.01 slider grid. For per-sample models, a hollow star marks the exact optimum over all observed thresholds whenever the grid can't reach it (probabilities compressed near 0 or 1). Own-cohort and transfer use the actual per-sample predictions (exact confusion counts). Combined uses the pooled 100×10 bootstrap out-of-fold predictions on the 0.01 grid (rates only; one sample appears many times). CV-based cutoffs are illustrative operating points, not validated deployment thresholds. Only the transfer basis is an independent cohort.
Data explorer
All AUC rows across own-cohort, transfer and combined analyses and both FS methods. Click a header to sort. miRNAs downloads the ranked miRNA list of the full-data model behind that row. Own-cohort and transfer rows give the source cohort's whole-data GLM (the exact model behind the transfer AUCs; own-cohort AUC is that procedure's 10-fold CV), non-zero coefficients. Combined rows give the final combined model: GLM non-zero coefficients, RF MeanDecreaseGini, or XGB input features (unranked; the saved boosters can't be reloaded). The cov/both batch indicators are listed last as
batch_covariate.