With global climate change accelerating, marine ecosystems face the challenge of adapting to a persistently changing environment. Species well established in the current climate might be outcompeted by species more successful in a warmer climate, thereby affecting ecosystem trophic structure and their role within the future global carbon cycle. How well marine communities will be able to adapt depends on their biodiversity. The observed biodiversity, however, often is not captured in ecosystem models employed to understand and predict consequences of climate change. In these models, zooplankton predation is often greatly simplified in spite of its potential to govern plankton dynamics: characteristics like feeding strategies and community responses are rarely considered in detail. Here, we compare the effects of different predation formulations on phytoplankton diversity in a global ecosystem model. In a model ocean seeded with several dozens of competing phytoplankton species, a small number of species dominate when employing a commonly used simplistic predation formulation. Diversity increases if the response of the entire zooplankton community on prey composition is considered. Combining this community response with a mechanistic predation model more than doubles the number of coexisting species. Employing a sound predation formulation is a prerequisite for successfully estimating the role and response of marine ecosystems in a future climate.