Change search
ReferencesLink to record
Permanent link

Direct link
De-identifying Swedish clinical text - refinement of a gold standard and experiments with Conditional random fields
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.
2010 (English)In: Journal of Biomedical Semantics, ISSN 2041-1480, Vol. 1:6Article in journal (Refereed) Published
Abstract [en]


In order to perform research on the information contained in Electronic Patient Records (EPRs), access to the data itself is needed. This is often very difficult due to confidentiality regulations. The data sets need to be fully de-identified before they can be distributed to researchers. De-identification is a difficult task where the definitions of annotation classes are not self-evident.


We present work on the creation of two refined variants of a manually annotated Gold standard for de-identification, one created automatically, and one created through discussions among the annotators. The data is a subset from the Stockholm EPR Corpus, a data set available within our research group. These are used for the training and evaluation of an automatic system based on the Conditional Random Fields algorithm. Evaluating with four-fold cross-validation on sets of around 4-6 000 annotation instances, we obtained very promising results for both Gold Standards: F-score around 0.80 for a number of experiments, with higher results for certain annotation classes. Moreover, 49 false positives that were verified true positives were found by the system but missed by the annotators.


Our intention is to make this Gold standard, The Stockholm EPR PHI Corpus, available to other research groups in the future. Despite being slightly more time-consuming we believe the manual consensus gold standard is the most valuable for further research. We also propose a set of annotation classes to be used for similar de-identification tasks.

Place, publisher, year, edition, pages
BioMed Central , 2010. Vol. 1:6
National Category
Information Science
Research subject
Computer and Systems Sciences
URN: urn:nbn:se:su:diva-51880DOI: 10.1186/2041-1480-1-6OAI: diva2:386347
Available from: 2011-01-12 Created: 2011-01-12 Last updated: 2011-02-09Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textfulltext

Search in DiVA

By author/editor
Dalianis, HerculesVelupillai, Sumithra
By organisation
Department of Computer and Systems Sciences
Information Science

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 34 hits
ReferencesLink to record
Permanent link

Direct link