All living organisms have a “membrane proteome” that mainly consists of α-helical mem- brane proteins containing one or more TM-helices. Prediction methods have been extensively used to identify as well as to classify the topology of these proteins. For current state-of-the- art methods, the prediction of correct topology of membrane proteins has been reported to be above 80%. However, this performance has only been observed in small and possibly biased datasets. Here, we add four “genome-scale” datasets, including a recent large set of experimen- tally validated membrane proteins with glycosylation sites. This set is also used to examine whether the qualities of topology predictions hold and if any prediction methods perform con- sistently better than others. We find that methods utilizing multiple sequence alignments are overall superior to methods that do not. The best performance is obtained by TOPCONS, a consensus method which combines several of the other prediction methods. Further, we show that the accuracy is most likely lower in eukaryotes than for prokaryotic proteins as the agree- ment between the predictors is significantly lower there. Finally, we show that three related methods, Phobius, Phillius and PolyPhobius, that incorporate a specific signal peptide module are superior to all other methods at the task of distinguishing between membrane and non- membrane proteins.