This is a false dichotomy. We are frugivores biologically so you are right, but they are also right because meat played an important part in the evolution of our large brains. For most of our evolutionary history we did not eat very much meat. We ate like chimpanzees and bonobos do today....mostly fruit, with some insects and small animals when they could be caught. A lot of meat in the diet is a more recent development of our homonid ancestors, but our overall anatomy is still that of a frugivore.
ETA: With the advent of dietary supplements, we have the option to not exploit animals. To continue to use them when it isn't necessary is to needlessly kill animals, which is immoral. For them to say that it is moral, or that we 'should' because it was an important part of our development as a species, is a non-sequitur. Breast milk/infant formula is an important part of our development as individuals but that doesn't mean we should consume it into adulthood. Similarly, the role that meat played in our development as a large brained species doesn't mean we should consume it today. Our brains are already as big as they are going to get. The next step in our evolution will probably be some transhumanistic merging of biology and technology such as the introduction of nanobots into our bloodstream and brains that will give us instant access to information without having to use a computer or other device, ala Ray Kurzweil et al.