LIGO and Virgo's third observing run revealed the first neutron star-black hole (NSBH) merger candidates in gravitational waves. These events are predicted to synthesize r-process elements(1,2)creating optical/near-infrared 'kilonova' emission. The joint gravitational wave and electromagnetic detection of an NSBH merger could be used to constrain the equation of state of dense nuclear matter(3), and independently measure the local expansion rate of the Universe(4). Here, we present the optical follow-up and analysis of two of the only three high-significance NSBH merger candidates detected to date, S200105ae and S200115j, with the Zwicky Transient Facility(5). The Zwicky Transient Facility observed similar to 48% of S200105ae and similar to 22% of S200115j's localization probabilities, with observations sensitive to kilonovae brighter than -17.5 mag fading at 0.5 mag d(-1)in the g- and r-bands; extensive searches and systematic follow-up of candidates did not yield a viable counterpart. We present state-of-the-art kilonova models tailored to NSBH systems that place constraints on the ejecta properties of these NSBH mergers. We show that with observed depths of apparent magnitude similar to 22 mag, attainable in metre-class, wide-field-of-view survey instruments, strong constraints on ejecta mass are possible, with the potential to rule out low mass ratios, high black hole spins and large neutron star radii.