The heat shock protein 90 (Hsp90) is a molecular chaperone central to client protein folding and maturation in eukaryotic cells. During its chaperone cycle, Hsp90 undergoes ATPase-coupled large-scale conformational changes between open and closed states, where the N-terminal and middle domains of the protein form a compact dimerized conformation. However, the molecular principles of the switching motion between the open and closed states remain poorly understood. Here we show by integrating atomistic and coarse-grained molecular simulations with small-angle X-ray scattering experiments and NMR spectroscopy data that Hsp90 exhibits rich conformational dynamics modulated by the charged linker, which connects the N-terminal with the middle domain of the protein. We show that the dissociation of these domains is crucial for the conformational flexibility of the open state, with the separation distance controlled by a beta-sheet motif next to the linker region. Taken together, our results suggest that the conformational ensemble of Hsp90 comprises highly extended states, which could be functionally crucial for client processing.