A large threat to biological diversity worldwide is habitat fragmentation, with negative effects on organisms arising as a result of changes in the abiotic or biotic environment and via changes in genetic processes. We examined genetic diversity, population structure and gene flow as well as the importance of genetic diversity for plant and population performance in 17 differently-sized populations of the locally rare, self-incompatible, perennial forest herb Phyteuma spicatum. Genetic variation, characterized by AFLPs, was low (mean gene diversity, H_e: 0.126) and generally increased with increasing population size. Bayesian analysis of population structure showed that populations located in the same forest patch or in close vicinity formed genetic clusters. Population differentiation conformed to a pattern of isolation by distance (IBD). Gene dispersal distances were higher when estimated within than among populations, suggesting that gene flow is more restricted on a landscape scale. Two seedling vital rates were influenced by population genetic parameters, while most late life-cycle stages and population growth rate were not affected. These results suggest that genetic load is purged early in the life cycle by selective mortality, which may buffer populations against the negative genetic effects of small population size. The overall low levels of genetic diversity, however, imply that populations are poorly equipped to respond to future changes in the environment. This is of special concern as short-distance gene dispersal indicates that colonization of new sites in response to putative changes in the environment is unlikely.