Attending to emotion can shed light on why recognizing an ethical issue and taking responsibility for it can be so demanding. To examine emotions related to taking or not taking responsibility for ethical action, we conducted a semi-structured interview study with 23 individuals working in interaction design and developing AI systems in Scandinavian countries. Through a thematic analysis of how participants attribute ethical responsibility, we identify three ethical stances, that is, discursive approaches to answering the question 'who should act': an individualized I-stance ("the responsibility is mine"), a collective we-stance ("the responsibility is ours"), and a distanced they-stance ("the responsibility is someone else's"). Further, we introduce the concepts of distancing and vulnerability to analyze the emotion work that these three ethical stances place on technology practitioners in situations of low- and high-scale technology development, where they have more or less control over the outcomes of their work. We show how the we- and they-stances let technology practitioners distance themselves from the results of their activity, while the I-stance makes them more vulnerable to emotional and material risks. By illustrating the emotional dimensions involved in recognizing ethical issues and embracing responsibility, our study contributes to the field of Ethics in Practice. We argue that emotions play a pivotal role in technology practitioners' decision-making process, influencing their choices to either take action or refrain from doing so.