We examine learning dynamics in deep recurrent networks, focusing on the behavior near the boundary in the depth-width plane separating under- from overparametrized networks, known as the interpolation transition. The training data are Bach chorales in four-part harmony, and the learning is by stochastic gradient descent with a cross-entropy loss function. We find critical slowing down of the learning, approaching the transition from the overparametrized side: For a given network depth, learning times to reach small training loss values appear to diverge proportional to 1/(w-wc) as the width w approaches a (loss-dependent) critical value wc. We identify the zero-loss limit of this value with the interpolation transition. We also study aging (the slowing down of fluctuations as the time since the beginning of learning increases). Taking a system that has been learning for a time τw, we measure the subsequent mean-square fluctuations of the weight values at times τ>τw. In the underparametrized phase, we find that they are well-described by a single function of τ/τw. While this scaling holds approximately at short times at the transition and in the overparametrized phase, it breaks down at longer times when the training loss gets close to the lower limit imposed by the stochastic gradient descent dynamics. Both this kind of aging and the critical slowing down are also found in certain spin glass models, suggesting that those models contain the most essential features of the learning dynamics.

Realized covariance matrices are often constructed under the assumption that richness of intra-day return data is greater than the portfolio size, resulting in nonsingular matrix measures. However, when for example the portfolio size is large, assets suffer from illiquidity issues, or market microstructure noise deters sampling on very high frequencies, this relation is not guaranteed. Under these common conditions, realized covariance matrices may obtain as singular by construction. Motivated by this situation, we introduce the Singular Conditional Autoregressive Wishart (SCAW) model to capture the temporal dynamics of time series of singular realized covariance matrices, extending the rich literature on econometric Wishart time series models to the singular case. This model is furthermore developed by covariance targeting adapted to matrices and a sector wise BEKK-specification, allowing excellent scalability to large and extremely large portfolio sizes. Finally, the model is estimated to a 20-year long time series containing 50 stocks and to a 10-year long time series containing 300 stocks, and evaluated using out-of-sample forecast accuracy. It outperforms the benchmark models with high statistical significance and the parsimonious specifications perform better than the baseline SCAW model, while using considerably less parameters. 

Background: The t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm has emerged as one of the leading methods for visualising high-dimensional (HD) data in a wide variety of fields, especially for revealing cluster structure in HD single-cell transcriptomics data. However, t-SNE often fails to correctly represent hierarchical relationships between clusters and creates spurious patterns in the embedding. In this work we generalised t-SNE using shape-aware graph distances to mitigate some of the limitations of the t-SNE. Although many methods have been recently proposed to circumvent the shortcomings of t-SNE, notably Uniform manifold approximation (UMAP) and Potential of heat diffusion for affinity-based transition embedding (PHATE), we see a clear advantage of the proposed graph-based method.

Results: The superior performance of the proposed method is first demonstrated on simulated data, where a significant improvement compared to t-SNE, UMAP and PHATE, based on quantitative validation indices, is observed when visualising imbalanced, nonlinear, continuous and hierarchically structured data. Thereafter the ability of the proposed method compared to the competing methods to create faithfully low-dimensional embeddings is shown on two real-world data sets, the single-cell transcriptomics data and the MNIST image data. In addition, the only hyper-parameter of the method can be automatically chosen in a data-driven way, which is consistently optimal across all test cases in this study.

Conclusions: In this work we show that the proposed shape-aware stochastic neighbor embedding method creates low-dimensional visualisations that robustly and accurately reveal key structures of high-dimensional data.

In this paper the concept of quantile-based optimal portfolio selection is introduced and a specific portfolio connected to it, the conditional value-of-return (CVoR) portfolio, is proposed. The CVoR is defined as the mean excess return or the conditional value-at-risk (CVaR) of the return distribution. The portfolio selection consists solely of quantile-based risk and return measures. Financial institutions that work in the context of Basel 4 use CVaR as a risk measure. In this regulatory framework sufficient and necessary conditions for optimality of the CVoR portfolio are provided under a general distributional assumption. Moreover, it is shown that the CVoR portfolio is mean-variance efficient when the returns are assumed to follow an elliptically contoured distribution. Under this assumption the closed-form expression for the weights and characteristics of the CVoR portfolio are obtained. Finally, the introduced methods are illustrated in an empirical study based on monthly data of returns on stocks included in the S&P index. It is shown that the new portfolio selection strategy outperforms several alternatives in terms of the final investor wealth.

In this paper we present several goodness-of-fit tests for the centralized Wishart process, a popular matrix-variate time series model used to capture the stochastic properties of realized covariance matrices. The new test procedures are based on the extended Bartlett decomposition derived from the properties of the Wishart distribution and allows to obtain sets of independently and standard normally distributed random variables under the null hypothesis. Several tests for normality and independence are then applied to these variables in order to support or to reject the underlying assumption of a centralized Wishart process. In order to investigate the influence of estimated parameters on the suggested testing procedures in the finite-sample case, a simulation study is conducted. Finally, the new test methods are applied to real data consisting of realized covariance matrices computed for the returns on six assets traded on the New York Stock Exchange.

In this paper we derive the finite-sample distribution of the estimated weights of the tangency portfolio when both the population and the sample covariance matrices are singular. These results are used in the derivation of a statistical test on the weights of the tangency portfolio where the distribution of the test statistic is obtained under both the null and alternative hypotheses. Moreover, we establish the high-dimensional asymptotic distribution of the estimated weights of the tangency portfolio when both the portfolio dimension and the sample size increase to infinity. The theoretical findings are implemented in an empirical application dealing with the returns on the stocks included into the S&P 500 index.

We study a biological autoregulation process, involving a protein that enhances its own transcription, in a parameter region where bistability would be present in the absence of fluctuations. We calculate the rate of fluctuation-induced rare transitions between locally stable states using a path integral formulation and Master and Chapman-Kolmogorov equations. As in simpler models for rare transitions, the rate has the form of the exponential of a quantity S-0 (a barrier) multiplied by a prefactor eta. We calculate S-0 and eta first in the bursting limit (where the ratio gamma of the protein and mRNA lifetimes is very large). In this limit, the calculation can be done almost entirely analytically, and the results are in good agreement with simulations. For finite gamma numerical calculations are generally required. However, S-0 can be calculated analytically to first order in 1/gamma, and the result agrees well with the full numerical calculation for all gamma > 1. Employing a method used previously on other problems, we find we can account qualitatively for the way the prefactor eta varies with gamma, but its value is 15-20% higher than that inferred from simulations.

We propose a new algorithm for inferring the state of hidden spins and reconstructing the connections in a synchronous kinetic Ising model, given the observed history. Focusing on the case in which the hidden spins are conditionally independent of each other given the state of observable spins, we show that calculating the likelihood of the data can be simplified by introducing a set of replicated auxiliary spins. Belief propagation (BP) and susceptibility propagation (SusP) can then be used to infer the states of hidden variables and to learn the couplings. We study the convergence and performance of this algorithm for networks with both Gaussian-distributed and binary bonds. We also study how the algorithm behaves as the fraction of hidden nodes and the amount of data are changed, showing that it outperforms the Thouless-Anderson-Palmer (TAP) equations for reconstructing the connections.

We derive learning rules for finding the connections between units in stochastic dynamical networks from the recorded history of a visible subset of the units. We consider two models. In both of them, the visible units are binary and stochastic. In one model the hidden units are continuous-valued, with sigmoidal activation functions, and in the other they are binary and stochastic like the visible ones. We derive exact learning rules for both cases. For the stochastic case, performing the exact calculation requires, in general, repeated summations over an number of configurations that grows exponentially with the size of the system and the data length, which is not feasible for large systems. We derive a mean field theory, based on a factorized ansatz for the distribution of hidden-unit states, which offers an attractive alternative for large systems. We present the results of some numerical calculations that illustrate key features of the two models and, for the stochastic case, the exact and approximate calculations.

Heterogeneous and dynamic single cell migration behaviours arise from a complex multi-scale signalling network comprising both molecular components and macromolecular modules, among which cell-matrix adhesions and F-actin directly mediate migration. To date, the global wiring architecture characterizing this network remains poorly defined. It is also unclear whether such a wiring pattern may be stable and generalizable to different conditions, or plastic and context dependent. Here, synchronous imaging-based quantification of migration systemorganization, represented by 87 morphological and dynamic macromolecular module features, and migration system behaviour, i.e., migration speed, facilitated Granger causality analysis. We thereby leveraged natural cellular heterogeneity to begin mapping the directionally specific causal wiring between organizational and behavioural features of the cell migration system. This represents an important advance on commonly used correlative analyses that do not resolve causal directionality. We identified organizational features such as adhesion stability and adhesion F-actin content that, as anticipated, causally influenced cell migration speed. Strikingly, we also found that cell speed can exert causal influence over organizationalfeatures, including cell shape and adhesion complex location, thus revealing causality in directions contradictory to previous expectations. Importantly, by comparing unperturbed and signalling-modulated cells, we provide proof-of-principle that causal interaction patterns are in fact plastic and context dependent, rather than stable and generalizable.