Deep bidirectional transformers for discovering latent knowledge in medical papers on Acute Myeloid Leukemia

Abstract

The volume of information produced and accessed through the Internet is large and growing. Consequently, the amount of information available in scientific papers follows the same trend, making manual analysis of all existing content unfeasible. Several strategies and architectures for artificial neural networks have emerged to represent texts using dense vectors, called word vectors. These techniques are continually evolving and are capable of processing increasingly more significant sets of texts with less computational resources. They allow for training huge text representation models for specific knowledge domains, as is the case of PubMedBERT. Using a large corpus of a single area, the model can better capture the relationships between words. When creating representation models from prefaces of scientific papers in materials science, Tshitoyan et al. (2019) observed that the knowledge of certain relationships between elements was latent. Furthermore, they demonstrated that some relationships found in the texts were discovered only years ahead. In this context, this research project proposes to train word vectors from scientific papers about a specific disease to capture and analyze whether it is possible to obtain latent knowledge that can accelerate the discovery of new diagnoses, prognoses, and treatments. (AU)