Why not? I'm not saying you are wrong here, but this is a central, if not THE central tent pole supporting your argument against current notions of "evolutionary psychology", and as such needs more support than any of the conclusions derived from it.
The existence in the fossil record of "Punctuated equilibrium" as a feature of the origin of species suggests that rapid evolutionary changes may be closer to the norm than are gradual changes, and if this is true then timescapes of ten thousand years or less may be entirely adequate for significant evolutionary changes. They may even be the usual scale of time for species change in response to environmental changes.
But even if we take longer adaptive periods into consideration, and particularly if we include primate behaviors into such models, we certainly have to account for sexual dimorphism in such models.
Looking at the difference between sexually promiscuous species like Bonobos and more sexually exclusive species like Gorillas we see a close association between the degree of dimorphism and the degree of promiscuity, with the Bonobos having a very limited degree of dimorphism and significant promiscuity, and the male dominated polygamous gorilla groups having the largest degree of dimorphism and limited promiscuity.
While it's hard to argue that there is any direct genetic or evolutionary link between the practices and physical differences of these two species and humans, it's also hard to ignore the apparent connection between male size and aggression and male sexual exclusivity. Further, it's hard to ignore the connection between relative testosterone levels and aggressive domination behaviors.
When we compare humans with these primate examples it's hard not to notice the significant sexual dimorphism in humans and the significant differences in testosterone levels between the human sexes. If we are to claim that humans are NOT like many other species where males are significantly larger and more aggressive and also compete for sexual access while assembling polygamous social groups we need to explain why they are different.
