Three-seed means · AP descending
- Ours · AP
- — Primary ranking metric
- ROC-AUC
- — Threshold-independent discrimination
- MCC
- — Classification at a fixed threshold
- AP mean difference vs best baseline
- — —Means only; significance not assessed
Swipe the table horizontally for all metrics. The model column stays visible.
Evaluation protocol ↓Score and ranking evidence
Fixed reference samples; independent of the dataset selected aboveCSD test set · reference seed 4222,168 entities · 7,464 positive labels
0.000.250.500.751.00
Hover or focus a bar to inspect its counts
Positive labelsNegative labelsThreshold 0.584
This reference distribution contextualizes individual scores. Scores are not calibrated probabilities; a low score does not rule out a symmetry-allowed crystal form.
CSD validation set · Ours vs D-MPNN22,168 entities · 7,356 positives
Ours · AP —D-MPNN · AP —Positive rate —
Both curves use the validation set because aligned test predictions are unavailable for D-MPNN. These AP values are separate from the test-set model comparison above.
Evaluation protocol and interpretation
How to read the evidence
- Primary metric
- Average precision (AP); compare models within the same dataset and sample scope.
- Threshold
- 0.583931… (displayed as 0.584), selected by maximum validation MCC. COD reuses it without refitting or threshold search.
- Seeds
- Comparison metrics are three-seed means (42, 2026, 3407). Single-molecule scores use seed 42; seed spread is max − min, not a standard deviation.
- Sample scope
- The comparison table uses common model coverage: CSD 22,159; high-quality subset 15,554; COD 1,120. The full test histogram has 22,168 entities. PR curves use the validation set.
- Uncertainty
- The bundled comparison contains means only; no per-model standard deviations or significance claims are added.
What the score supports
Rank molecular candidates for symmetry-allowed piezoelectric screening, then prioritize experimental follow-up using applicability and seed agreement.
- It is not a calibrated probability, d33 prediction or guarantee of a measurable experimental signal.
- Molecular structure alone does not determine crystal packing or every polymorph.
- Stereochemistry is excluded from model input; stereoisomers receive the same model score.
- Applicability should be checked for salts, cocrystals, metal complexes and structures outside the training distribution.