Abstract:
Graph-based learning methods have gained more attention in colorectal adenocarcinoma cancer (CRA) grading tasks for encoding the tissue structure information, which patch-wise CNN based methods fail to. Graph-based methods usually involve extracting nuclei features in the histology images as cell-graph node features and modeling the connections between nodes to construct cell-graphs. However, it is infeasible to directly train a classification model to extract nuclei features as we normally do in nature images since different types of nuclei often cluster together. We propose a Masked Nuclei Patch (MNP) approach to train a ResNet-50 as a strong feature encoder to extract more representative nuclei feature for enhancing the overall performance. Graph Neural Networks (GNNs) are often used to train cell-graphs for different tasks. But GNN may struggle to capture the long-range dependency due to its underlying recurrent structure. Therefore, we propose a new network architecture named Hierarchical Transformer GraphNeural Network, which merits both GNN and Transformer, as a strong competitor for CRA grading tasks. We have achieved the state-of-the-art results on two publicly available CRA grading datasets: the colorectal cancer (CRC) dataset (98.55%) and the extended colorectal cancer (Extended CRC) dataset (95.33%).
