Research Note
Domain Pretraining for Heart Localization
A follow-up to our real-time detection proof of concept: we tested a promising idea for improving heart localization on our small training set. The result was negative, and we are publishing it anyway.
Six days after publishing this report, a review of our own data turned up two errors in it. We are correcting them here in public rather than quietly editing the page, because a correction that nobody sees is not a correction.
1. We overstated the size of our training set by about six times. This report says our heart localizer was trained on roughly 1,800 images. The real number is 267. We had been describing one dataset as if it were two, and a count of 1,500 files had been read as 1,500 images, when those files were actually 300 images plus their annotation files. The experiment itself was run correctly and the results below are accurate. What was wrong was our description of how much data stood behind them.
2. This experiment was probably not sensitive enough to answer its own question. With only about 53 images used for scoring in each round, the natural variation in our measurements is large enough that it could hide the very improvement we were looking for. Our conclusion below, that pretraining did not measurably help, is still correct as stated. But we can no longer say why with confidence. "Pretraining does not work here" and "our measurement was too coarse to tell" are, on this evidence, the same picture.
We are now running a follow-up experiment specifically to establish how small a difference our testing is actually capable of detecting, before we spend more effort on model improvements we might not be able to see.
Background: Our fetal heart localizer is trained on a small set of four-chamber-view (4CV) ultrasound images. When first published, this sentence said "roughly 1,800" images. The correct figure is 267. See the correction above. Small training sets are a known limiter of model reliability, and one common mitigation in the research literature is domain pretraining: training a model first on a larger, related pool of images so it learns general visual patterns before it ever sees the small target dataset.
Objective: We tested whether pretraining our localizer on FPUS23, a public fetal-ultrasound phantom dataset with roughly 15,700 images of general body regions (head, abdomen, arm, legs, no cardiac content), would improve localization accuracy on our cardiac task compared to our existing baseline.
Before running the experiment, we wrote down our method, our comparison metric, and the criteria for what would count as a positive, null, or negative result. We committed to publishing the outcome regardless of which one it turned out to be.
We used a two-stage approach. First, we trained our YOLO11n localizer on FPUS23's general bounding boxes to produce a fetal-ultrasound-adapted starting checkpoint. Second, we fine-tuned that checkpoint on our existing cardiac dataset, using the exact same hyperparameters, image splits, and 5-fold cross-validation as our original baseline, changing only the starting weights. This isolates pretraining as the single variable being tested.
All data used is public and de-identified. FPUS23 is a phantom dataset, meaning the images are of a manufactured model, not real patients.
Results
| Metric | Baseline | FPUS23-pretrained |
|---|---|---|
| mAP@50 (mean, 5-fold CV) | 0.582 ± 0.141 | 0.550 ± 0.137 |
| Precision (mean) | 0.711 ± 0.089 | 0.678 ± 0.126 |
| Recall (mean) | 0.648 ± 0.099 | 0.603 ± 0.075 |
Key Finding
The FPUS23-pretrained model performed slightly lower on every metric than our existing baseline, but the difference is well within the natural variation we see between folds in our 5-fold cross-validation. By the criteria we set before running the experiment, this is a null result: pretraining on FPUS23, in this form, did not measurably improve our heart localizer. We are not adopting the pretrained checkpoint, and our existing baseline model remains the one in use.
Why the Result Was Likely Negative
A few plausible reasons, none of which we have separately tested yet: FPUS23's images are of a manufactured phantom rather than real tissue, so the visual patterns it teaches may not transfer as well as pretraining on real ultrasound would. Its bounding boxes also cover large, whole-body regions like the abdomen or a limb, a different and easier problem than locating the much smaller, more specific structures of the heart. It is also possible that our fine-tuning process, unchanged from the original baseline for the sake of a fair comparison, simply overwrote most of what the pretraining step contributed.
Added in the July 28 correction: there is a fourth possibility we did not consider at the time, and it may be the most likely one. Our measurement itself may simply have been too coarse. We compared two averages built from five scores each, and every one of those scores came from about 53 images. That approach throws away most of the information available and is easily swamped by which particular images happened to land in which group. The effect we were hunting for could have been real and still invisible to us.
Why We Publish Negative Results
Part of building a trustworthy AI tool for a use as important as CHD screening is being honest about what does and does not work, not just what looks good. A negative result rules out an appealing but ineffective shortcut and keeps our public claims grounded in what we have actually verified. We would rather share a disappointing finding than stay quiet about it.
The same reasoning applies to the correction at the top of this page. We found those errors ourselves, in our own data, and nobody would have known if we had fixed the number quietly. Publishing the correction costs us something in the short term. Not publishing it would cost us the only thing that makes the rest of our research worth reading.
Next Steps
Our immediate next step, added as a direct result of the correction above, is an experiment on our own measurement process: we want to establish the smallest difference our testing can reliably detect, so that we know which of our results are meaningful and which are noise. Until that is settled, we are holding off on further model improvements we may not be able to measure.
After that, we will retest the idea behind this experiment using real tissue images rather than phantom ones, and we will expand our training set beyond its current 267 images. Our top priority overall remains validating our image quality signal against real expert ratings, described in our proof-of-concept report. We write each of these experiments down in advance, including what would count as failure, before we run them.