Segmenting and Classifying Clinical Notes
NLP + Topological Data Analysis on MIMIC-III
Data Engineering - University of Tennessee
BioClinicalBERT, Doc2Vec, TF-IDF, Mapper (TDA), Hierarchical Clustering
Segmenting and Classifying Clinical Notes
Turning 2.3 million unstructured MIMIC-III note segments into a labeled dataset, then classifying and clustering them with document embeddings and Topological Data Analysis.
Course: COSC 526 – Data Engineering, University of Tennessee, Knoxville
Additional Resources: Abstract · Poster (PDF) · Git Repository · Dataset (MIMIC-III)
Electronic Health Records contain large amounts of clinical notes, but their unstructured, inconsistent formatting makes it hard for clinicians to find and interpret important information, especially when a single record is filled in by multiple people across different departments. Supervised approaches can classify note segments, but they depend on manual annotation, which is expensive to produce at scale.
This project tackles that problem using the publicly available MIMIC-III dataset, focusing on its "Nursing/other" note category - 822,497 unique notes. Using regex-based NLP techniques, I segmented that raw text into a structured dataset of 2.3 million labeled segments across 2,121 unique segment titles. From there, I extracted two complementary embeddings; sparse TF-IDF + SVD features to train a supervised segment-title classifier, and dense Doc2Vec embeddings to unsupervised-cluster the label space itself, using both Hierarchical Agglomerative Clustering and the Topological Data Analysis (TDA) Mapper algorithm.
From raw notes to organized labels
Four stages take unstructured nursing notes to a classified, clustered segment-label space.
Regex-Based Segmentation
Subsections are located by special-character patterns, then cleaned: stripping stray whitespace, dates, and unwanted headings, and normalizing all numbers to a static token so numeric variation doesn't fragment otherwise-identical segments.
Dual Embeddings
Each segment is represented two ways: TF-IDF reduced with SVD into sparse vectors for supervised classification, and Doc2Vec dense embeddings for unsupervised clustering of the label space.
Segment Classification
A neural classifier trained on the TF-IDF + SVD features predicts a segment's title from its body text, evaluated with F1 scores across all 2,121 segment types.
Label-Space Clustering
Segment titles are grouped by average embedding similarity using Hierarchical Agglomerative Clustering (k=72, chosen via silhouette score) and the TDA Mapper algorithm, which builds a network directly from the embedding space.
Built with
What the classifier and the clustering found
Two separate questions: how well can a model predict a segment's title, and does the label space itself have usable structure?
Does classification accuracy hold up across all 2,121 segment titles?
Segment types vary enormously in frequency, a handful like 'resp' and 'neuro' occur well over 100,000 times each, while most of the 2,121 titles are comparatively rare.
Solution
High-frequency segment types reached higher and more stable F1 scores; rare, low-example segment types scored lower and far more variably. Most segment types had low F1 overall; data imbalance, not the embedding or model choice, is the dominant bottleneck.
Can unsupervised methods reveal genuine structure in 2,121 raw segment titles?
Many of the raw titles are near-duplicates or inconsistent abbreviations of the same underlying concept, with no existing hierarchy to merge them.
Solution
Both the Mapper graph and hierarchical clustering (k=72, chosen by silhouette score) surfaced tightly connected clusters of co-occurring, semantically related titles, with rare or highly specific labels showing up as peripheral or isolated nodes; a usable path toward merging redundant labels.
The clearest takeaway is that data imbalance, not model capacity, limits classification quality on rare segment types; motivating label merging, better clustering, or data augmentation as the next steps to make note-segment classification reliable across the full label space, not just the common ones.
Figures
