The macromolecular conformation of the constituent polysaccharides in lignocellulosic biomass influences their supramolecular interactions, and therefore their function in plants and their performance in technical products. The flexibility of glycosidic linkages from the backbone of hemicelluloses was studied by evaluating the conformational freedom of the φ and ψ dihedral angles using molecular dynamic simulations, additionally selected molecules were correlated with experimental data by NMR spectroscopy. Three types of β-(1→4) glycosidic linkages involving the monosaccharides (Glcp, Xylp and Manp) present in the backbone of hemicelluloses were defined. Different di- and tetrasaccharides with combinations of such sugar monomers from hemicelluloses were simulated and free energy maps of the φ - ψ space and hydrogen bonding patterns were obtained. The glycosidic linkage between Glc-Glc or Glc-Man (C-type) was the stiffest with mainly one probable conformation; the linkage from Man-Man or Man-Glc (M-type) was similar but with an increased probability for an alternative conformation making it more flexible, and the linkage between two Xyl-units (X-type) was the most flexible with two almost equally populated conformations. Glycosidic linkages of the same type showed essentially the same conformational space in both disaccharides and in the central region of tetrasaccharides. Different probabilities of glycosidic linkage conformations in the backbone of hemicelluloses can be directly estimated from the free energy maps, which to a large degree affect the overall macromolecular conformations of these polymers. The information gained contributes to an increased understanding of hemicelluloses’ function both in the cell wall and in technical products.