Social-ecological resilience underlies popular sustainability concepts that have been influential in formulating the United Nations Sustainable Development Goals (SDGs), such as the Planetary Boundaries and Doughnut Economics. Scientific investigation of these concepts is supported by mathematical models of planetary biophysical and societal dynamics, both of which call for operational measures of resilience. However, current quantitative descriptions tend to be restricted to the foundational form of the concept: persistence resilience. We propose a classification of modern notions of social-ecological resilience from a multi-agent-environment perspective. This aims at operationalization in a complex systems framework, including the persistence, adaptation and transformation aspects of resilience, normativity related to desirable system function, first-vs. second-order and specific vs. general resilience. For example, we discuss the use of the Topology of Sustainable Management Framework. Developing the mathematics of resilience along these lines would not only make social-ecological resilience more applicable to data and models, but could also conceptually advance resilience thinking.