Routers have a speed setting for three reasons: safety, quality of cut, differences in wood species. Dialing in the right speed for the task at hand can save you a lot of time spent sanding. Safety is the primary concern, so lets talk about that first.
Safety
Larger router bits need to spin slower. The outer edge of a 3" router bit travels at well over 100 mph in most routers. A 1/2" bit clocks in at a pedestrian 30 mph. The larger bit also takes larger bites out of the wood. If the edge catches there is a lot more energy to be dissipated. Start your router at medium speed and work your way up.
Quality of cut
The bits we woodworkers use most of the time are too small to be concerned about safety when setting the speed of the router. Here the consideration is quality of cut. The router is either too slow, just right, or too fast.
The router is set too slow when it leaves behind bumps and scrapes. The bit is catching too much wood with each time. It tears the wood rather than cutting it. Speed up the router, or slow down your feed rate.
The router is set too fast when it creates tear out or leaves behind burn marks. The excess energy in the bit is transferred to the wood where it creates burns. Slow the router down or increase your feed rate.
Wood species
Hardwoods and soft woods machine differently. Softer woods require less energy to cut. Routers should be set slower when routing soft woods. Hardwoods are more robust and require faster router settings. The bit needs to chip away at the harder material in smaller chunks. Some hardwood species like cherry are susceptible to burning despite cherry being a hardwood. I recommend testing your router setting on a piece of scrap before you cut into unfamiliar lumber.