Automatic generation of the ground truth for tumor budding using H&E stained slides

Tumor budding (TB) is defined as a cluster of one to four tumor cells at the tumor invasive front. Though promising as a prognostic factor for colorectal cancer, its routine clinical use is hampered by high inter- and intra- observer disagreement on routine H&E staining. Pan-cytokeratin immunohistochemical staining increases agreement but is costly, nonroutine, and may yield tumor bud false positives. This makes the development of automatic algorithms to identify TB difficult. Therefore, we sought to develop an automated method to generate ground truth for tumor budding from routine H&E that benefits from pan-cytokeratin staining. Our database consisted of 1) 120 adjacent pan-cytokeratin and H&E tissue sections from which 648 manually-registered high-power fields were extracted and annotated for tumor regions and 2) ten adjacent sections from which 109 fields were extracted and annotated for tumor buds. Swin transformers were applied to segment tumor regions, resulting in 95% and 99% accuracy for tumor and non-tumor regions. This allowed for a narrow margin outside the tumor invasive front to identify TB. Next, we developed a novel method which registers pan-cytokeratin images to adjacent H&E images, transfers automatically-identified positive regions of pan-cytokeratin to H&E, then filters positive regions on H&E using size and number of nuclei. The average precision and recall were 0.3856 and 0.3254 on H&E when compared to pathologists on H&E, which exceeded 0.3470 and 0.1932 when comparing pathologists on H&E to pan-cytokeratin. Results provide preliminary evidence for the feasibility of our method to generate ground truth for TB using H&E.