The past two decades have witnessed an increasing interest in understanding and controlling materials at the pico- and femtosecond time scales, the so-called ultrafast regime. Among the broad field of condensed matter physics, magnetism and magnetic materials have attracted much interest both from a fundamental and an applied perspective. The field of ultrafast magnetism is at the frontier of current physics research, with fundamental questions that are still unanswered but which have the potential of impacting the data storage technology upon which our digitalized world relies on. Ultrafast lasers in the visible range (i.e., with energies in the eV range) have been widely used to study ultrafast magnetization dynamics, but due to the relatively large photon energy, they create highly non equilibrium states which tend to mask the fundamental coupling processes leading to ultrafast demagnetization. However, the relatively recent appearance of intense coherent terahertz (THz) radiation (with photon energies in the meV range) offers a new way to understand and manipulate the magnetic order, and is receiving much attention in the research community. As a major part of this thesis, a table-top experimental setup for generating intense THz radiation has been developed for the purpose of carrying out pump-probe studies of thin ferromagnetic metallic films. The setup is capable of delivering state-of-the-art THz electric fields as large as 1 MV/cm, corresponding to 0.3 T magnetic fields which can directly couple to the magnetization to trigger ultrafast dynamics. The ultrafast magnetization dynamics is probed with the time resolved magneto-optical Kerr effect with a resolution of approximately 40 fs. Three main scientific results have been obtained with this thesis work. First, the experimental evidence, in the form of a spin nutation in the THz range, of inertial magnetization dynamics in thin film ferromagnets, which we could describe with a modified version of the textbook Landau Lifshitz-Gilbert (LLG) equation to include a realistic inertial tensor. Second, with this modified LLG equation, we performed simulations to study the role of inertia in the switching of the magnetization with picosecond magnetic field pulses. We found that inertia leads to a so-called ballistic switching which is more robust to the details of the magnetic field pulse. Third, we studied the influence of crystalline order on the charge and spin transport at terahertz rates. We found that while the charge scattering follows the degree of crystalline order in the film, the spin scattering is enhanced at intermediate crystalline phases which have not fully ordered, but where the magnetic anisotropy is largest.