Humanity is facing many grand challenges at unprecedented rates, nearly everywhere, and at all levels. Yet virtually all these challenges can be traced back to the decision and behavior of autonomous agents that constitute the complex systems under such challenges. Agent-based modeling has been developed and employed to address such challenges for a few decades with great achievements and caveats. This article reviews the advances of ABM in social, ecological, and socio-ecological systems, compare ABM with other traditional, equation-based models, provide guidelines for ABM novice, modelers, and reviewers, and point out the challenges and impending tasks that need to be addressed for the ABM community. We further point out great opportunities arising from new forms of data, data science and artificial intelligence, showing that agent behavioral rules can be derived through data mining and machine learning. Towards the end, we call for a new science of Agent-based Complex Systems (ACS) that can pave an effective way to tackle the grand challenges.