It is widely thought that small populations should have less additive genetic variance and respond less efficiently to natural selection than large populations. Across taxa, we meta-analytically quantified the relationship between adult census population size (N) and additive genetic variance (proxy: h2) and found no reduction in h2 with decreasing N; surveyed populations ranged from four to one million individuals (1735 h2estimates, 146 populations, 83 species). In terms of adaptation, ecological conditions may systematically differ between populations of varying N; the magnitude of selection these populations experience may therefore also differ. We thus also meta-analytically tested whether selection changes with N, and found little evidence for systematic differences in the strength, direction, or form of selection with N across different trait types and taxa (7344 selection estimates, 172 populations, 80 species). Collectively, our results (i) indirectly suggest that genetic drift neither overwhelms selection more in small than in large natural populations, nor weakens adaptive potential/h2 in small populations, and (ii) imply that natural populations of varying sizes experience a variety of environmental conditions, without consistently differing habitat quality at small N. However, we caution that the data are currently insufficient to determine whether some small populations may retain adaptive potential definitively. Further study is required into (i) selection and genetic variation in completely isolated populations of known N, underrepresented taxonomic groups, and non-generalist species, (ii) adaptive potential using multidimensional approaches, and (iii) the nature of selective pressures for specific traits.