In this paper we discuss the impact to home, work, and civil life from the deployment of continuous listening services. An example service we call the Continuous Speech Stream (CSS), would provide a real time list of keywords generated from the user’s spoken interactions with others. Based on a user-study that engaged 10 users to record a full day of audio for processing into a sample stream, we report the concerns expressed by our participants on being misrepresented by their speech, unintentionally sharing sensitive data, and being unable to curate their presentation of self. We offer an initial set of recommendations for the design, testing, and deployment of IoT based services built on such rich, personal data.