The point is just that your MP3 player still plays MP3s in the same way, it doesn't make any difference if it can make phone calls too.
Firefly, the fact is that a decent DAC costs next to nothing. The "dedicated sound hardware" that you are talking about is the exact same stuff in phones and MP3 devices. You might hear a difference if you're using hi-fi gear, but then you wouldn't use the jack. The only real advantages to MP3 devices, which is why I use them, is that they usually have a way better interface, and a seperate battery (not draining your phone).
I have heard a difference in the past. There is a difference in quality from player to player. Plus the point you made about the interface and file management software.