Yeah. I replayed it recently and at some point they gave up and made it so you can get the berries in that cave without fighting any of the swarms.