Using "quick action" is not faster then hitting the numbers for a different weapon. My suggestion would be to make tappimg the # keys slower for switching weapons. Take for example, I select a handgun as my quickaction and my quickaction key is "v." Now it would be quicker for me to hit 2 then v. Also the weapon you are currently holding should just drop when you hit quickaction.
