Images in medical communication are appreciated by both professionals and patients, but we know little about how they are actually used in clinical practice and in the learning processes of patients. This qualitative study investigates the use and the meaning potential of two different types of heart images: the hand-drawn doctor’s sketch and the digital illustration from the web. The analysis starts with how these are recontextualised in social media, tracks them back to their original contexts and finally explores their material resources. The analytical perspective is that of social semiotics and multimodal discourse and interaction analysis. While the hand-drawn sketch is recontextualised as a witness of the specific consultation, the digital illustration is used to focus on the heart defect as such. It is also shown how the act of drawing works as a means for framing and structuring the consultation, slowing down the pace and reducing context and detail and thus focussing on what is uniquely relevant. While the digital and more realistic illustration is technically neutral and objective and is free from unique context, the drawing on paper is physically tied to its context of origin, which is also its main resource. 

Systemic-Functional Linguistics often appears as a complete and altogether exhaustive theory of language. If widened with a multimodal perspective, social semiotics seems to contain all you need – but does it? The research project Health literacy and knowledge-formation in the information society has investigated complex communication practices related to the pre-natal diagnosis of heart failure in children, pursuing linguistic purposes as well as educational and applied goals. Oral consultations, digital discussions, written texts and visual representations were studied across different contexts and epistemic cultures, from the medical consultation room to the private blog, using tools from SFL and social semiotics (e.g. genre analysis, systemic conversation analysis and Legitimation Code Theory) – but also tools from other traditions and disciplines (e.g. New Literacy Studies, Activity Theory and rhetoric). Based on examples from these studies, this paper discusses the strengths and limitations of SFL and social semiotics, both with respect to their potential to provide adequate analytical tools for complex and context-overarching meaning-making practices, and with respect to their compatibility with other relevant perspectives and disciplinary traditions, e.g. those of medical professionals.