Pretrained language models tailored to the target domain may improve predictive performance on downstream tasks. Such domain-specific language models are typically developed by pretraining on in-domain data, either from scratch or by continuing to pretrain an existing generic language model. Here, we focus on the latter situation and study the impact of the vocabulary for domain-adaptive pretraining of clinical language models. In particular, we investigate the impact of (i) adapting the vocabulary to the target domain, (ii) using different vocabulary sizes, and (iii) creating initial representations for clinical terms not present in the general-domain vocabulary based on subword averaging. The results confirm the benefits of adapting the vocabulary of the language model to the target domain; however, the choice of vocabulary size is not particularly sensitive with respect to downstream performance, while the benefits of subword averaging is reduced after a modest amount of domain-adaptive pretraining.