As a guy, and a new enough vegan (1.5 years) to have my recent omni past still relatively fresh in memory...
I can tell you that my libido ebbs and flows on these factors:
A) Enough calories.
B) Enough Zinc. (super important for health but especially for men)
C) Other broad spectrum nutrients including amino acids and Vitamin C. See here:
https://en.wikipedia.org/wiki/Semen#Human_semen
A) If you're not eating enough calories you're body is not going to have the urge to reproduce.
B) Low zinc intake is tied to sexual development problems in developing males, as well as low libido in developed males.
A considerable amount of zinc is also expelled in male semen.
Zinc is a major mineral tied to lots of interactions in the human body (male and female) but can be a concern for men just like iron can be a concern for women.
One of the concerns I had going vegan was getting enough Zinc. I had this fear because of the amount of alcohol I had been consuming. The body uses Zinc to detoxify alcohol, and I was afraid I might get deficient. However, this has not been the case eating an almost exclusive (besides alcohol) whole foods diet that includes plenty of zinc rich whole grains, seeds, nuts and legumes. Fruits and Vegetables are great in other ways but they do tend to be low on zinc.
As a 45 year old guy who drinks alcohol I can tell you that on a week to week basis (sometimes day to day) I have no libido issues. More alcohol, less libido. More calories from whole foods (especially seed type foods including whole
grains and legumes), more libido.
Flow issues are a related but separate issue. Men with erection problems may not lack necessary broad spectrum nutrients or calories, but their arteries are clogged with saturated fat, usually from years worth of eating too many animal products.
Some ex-Vegans who go back to eating animal products and claim instant, or near instant restoration of libido and other issues never seem to connect the dots on where those nutrients ultimately come from. For instance a cow eating grass or corn is getting all it's nutrients (including zinc) from these plants. Vitamins and minerals that play a part in human sexuality come from the plant kingdom ultimately, and animals are just vessels. However, don't expect much (or any in some cases) eating commercially prepared refined products.
One more point worth bringing up. I have heard some people parrot the idea that high mineral plant foods can be low mineral dependingon soil conditions (soil lacking minerals in general including zinc). I'm confident these people have never tried to grow, and bring to seed, plants. NPK is not enough for a plant to grow, and especially go to seed. I know someone in agribusiness (plant agribusiness) that tests soil and "prescribes" minerals and other products to help farmers raise the crops they are trying to raise. The simple fact is - without the right soil nutrients and conditions, plants will not grow properly and die prematurely.
In case you're wondering, I don't take vitamin, mineral or amino acid supplements. B12 from fortified low sodium Vegemite, that's it.