There are many differences of course. For instance, volume is one, as it is the size of the overall server vs. networking market.
However, thinking about it, a key difference lies in what is needed vs. what is possible, and whether they match. Confusing wording, isn't it? ...
With server virtualisation, the possible was joined with the needed. It was possible to run more workloads on existing hardware, meaning many servers were severely under-utilised, yet you still had to buy those servers. This was essentially (still is) a problem of the application and OS development really, for which a Hypervisor came to be a temporal solution.
So it was possible to run more workloads on existing CPU/memory footprint (meaning the horse power was there already) and it was required. Why was it required? To spend less money in server investments (and in network along the way too, for you need less ports to connect less servers, obviously), of course.
There was the big hit. An IT organisation could afford paying for a (somewhat) expensive virtualisation software because it would save you from buying a lot of hardware. Sure, on top of the virtualisation solution a lot more value was built soon afterwards, notably in HA and DR areas. But they payout was there from the need to use orders of magnitude less hardware. Oh, and you did not even had to change your preferred server vendor.
The same example is true for NIC virtualisation for instance. This is why technologies like HP VC with its virtual NICs or Cisco VIC have known success: you need to pay for less adapters. Same is true for FCoE to some extent. Essentially, you had idle capacity, and by applying a new technology you could make use of it, therefore investing less in hardware (be it servers, NICs, or whatever …).
Every customer, large and small, could see this clear economic benefit.
THAT is a key difference with network virtualisation. You do not have idle network capacity that you need to use to buy less network kit. You can not afford to buy less network capacity by simply adding network virtualisation (certainly, not with an overlay solution). You have to buy the same number of ports to connect your servers. Your traffic volume won't be less either, therefore you cannot afford to buy less powerful aggregation or core boxes to interconnect your access switches.
This is not to imply that network virtualisation does not bring any value. Some will be quick to point out agility, others scalability. Both arguments subject to debate, but in any case, none of them (a) applicable to every customer, and (b) having a distinct economic impact in every environment.