|And when you compare the same caliber in a handgun and a rifle, the short barrel and lower speed of the handgun needs a slower twist to get the bullet stabilized.|
Maybe you can compare it a little bit with burnrates of powder: a fast powder for a rifle will be slow or even very slow for a handgun or a shotgun.
Twist in a handgun barrel should be at least as fast as what it should be for a rifle barrel. If anything, it should be faster because the velocity will be less and thus the spin rate of the bullet in rpm will be less.
In regards to fast vs slow powders, it isn't really true that "fast" powders are best without some qualification. If you're referring to small capacity straight walled handgun cartridge cases, it's pretty much true. For those who shoot bottlenecked rifle cartridges in handguns, it's not true. The same loads that give good accuracy in rifles are generally the ones to use in handguns.
Essentially all of the powder, providing the pressure is adequate, burns up completely in a very short length of barrel whether the powder is "slow" or "fast." I know this to be true for "rifle powder" even down to the 5 inch length of handgun barrel.