[Python] Best NLP Approaches for Extracting Complex Variables from Unstructured Medical Notes...

I am working on extracting structured data from unstructured patient notes in electronic health records. I’ve been using SpaCy’s Named Entity Recognition models with an entity ruler (to define my desired keywords and their variants) and Negex for handling negations. This approach has worked well for short keywords and conditions (e.g., identifying terms like "perforation" or "pregnancy-induced hypertension"). However, I’m struggling with extracting longer, more complex variables, such as "age at first enteral feeds." From reading many notes, I’ve observed that the information is often present, but not expressed using a predictable set of terms. As a result, my current approach returns zero counts for these variables, which is inaccurate. What NLP methods would you recommend for extracting information that doesn’t rely on predefined keywords and rules? How can I capture these more nuanced data points from free text?

Continue reading...