The specific binding of oligonucleotide-tagged 100 nm magnetic nanoparticles (MNPs) to rolling circle products (RCPs) is investigated using our newly developed differential homogenous magnetic assay (DHMA). The DHMA measures ac magnetic susceptibility from a test and a control samples simultaneously and eliminates magnetic background signal. Therefore, the DHMA can reveal details of binding kinetics of magnetic nanoparticles at very low concentrations of RCPs. From the analysis of the imaginary part of the DHMA signal, we find that smaller MNPs in the particle ensemble bind first to the RCPs. When the RCP concentration increases, we observe the formation of agglomerates, which leads to lower number of MNPs per RCP at higher concentrations of RCPs. The results thus indicate that a full frequency range of ac susceptibility observation is necessary to detect low concentrations of target RCPs and a long amplification time is not required as it does not significantly increase the number of MNPs per RCP. The findings are critical for understanding the underlying microscopic binding process for improving the assay performance. They furthermore suggest DHMA is a powerful technique for dynamically characterizing the binding interactions between MNPs and biomolecules in fluid volumes.