The detailed observations of GW170817 proved for the first time directly that neutron star mergers are a major production site of heavy elements. The observations could be fit by a number of simulations that qualitatively agree, but can quantitatively differ (e.g., in total r-process mass) by an order of magnitude. We categorize kilonova ejecta into several typical morphologies motivated by numerical simulations, and apply a radiative transfer Monte Carlo code to study how the geometric distribution of the ejecta shapes the emitted radiation. We find major impacts on both spectra and light curves. The peak bolometric luminosity can vary by two orders of magnitude and the timing of its peak by a factor of five. These findings provide the crucial implication that the ejecta masses inferred from observations around the peak brightness are uncertain by at least an order of magnitude. Mixed two-component models with lanthanide-rich ejecta are particularly sensitive to geometric distribution. A subset of mixed models shows very strong viewing angle dependence due to lanthanide curtaining, which persists even if the relative mass of lanthanide-rich component is small. The angular dependence is weak in the rest of our models, but different geometric combinations of the two components lead to a highly diverse set of light curves. We identify geometry-dependent P Cygni features in late spectra that directly map out strong lines in the simulated opacity of neodymium, which can help to constrain the ejecta geometry and to directly probe the r-process abundances.