A leading hypothesis for the evolution of large brains in humans and other species is that a feedback loop exists whereby intelligent animals forage more efficiently, which results in increased energy intake that fuels the growth and maintenance of large brains. We test this hypothesis for the first time with high-resolution tracking data from four sympatric, frugivorous rainforest mammal species (42 individuals) and drone-based maps of their predominant feeding trees. We found no evidence that larger-brained primates had more efficient foraging paths than smaller brained procyonids. This refutes a key assumption of the fruit-diet hypothesis for brain evolution, suggesting that other factors such as temporal cognition, extractive foraging or sociality have been more important for brain evolution.