Based on your description, I think what you are hearing is the "EL84 Sizzle" Those power tubes have this grainy smooth tone. Also, most amps with those tubes are punchy with big lower mids. Hard to describe. It's a tone I like but not one I would want in every situation.
That's a good point. Regardless of which speaker I've tried in my Pro Junior (for me, nothing has beat an old Eminence Legend 1058 FWIW), the amp has a certain sizzle that isn't present in my 6v6 or 6L6 amps. It has great cleans in its own right, but it's never going to have the clarity of the BF amps.