My SW (necro build) stays on his mount at all times. I generally just use standard attack and let my minions and skeletal fort do the damage so it's not that cumbersome for me.
One drawback to it - when you walk into a cave, the game automatically dismounts you. However it leaves your mount right at the entrance of the cave. So when you come out and try to get back on it, you just get ported back to the cave. Sometimes it even ports your mount in the cave, and then the game get's all glitchy. Eg, see below