I would guess most people in the sub-$1000 player market are not hooking up via a digital interface. Both the quality of the DAC and the op amps in the player will make a difference in the quality of the signal leaving the analog out. Remember every CD player with analog out is essentially also a preamp. As a result you can certainly get better sound from a CD player by investing more in more than a $35 DVD player.
Eh.. all the stuff I was hooking it up with digitally 20 years ago was in the well under $500 range.
This stuff is all ancient dirt cheap technology today.
How would you feel if you screwed up in a blind A:B test and couldn't tell the difference between the $10 CD-ROM drive and the $1000 Audiophile marketed CD player? If they were both put through the same pre-amp/amp speakers you might not be able to tell, even if you took the analog feed out of the $10 player.
The other thing here is you could go buy a cheap CD player + HD audio digital player + a dedicated Pro-Audio/Audiophile DAC for less as well in some cases.
A lot of this effort/cost is probably better redirected into digitizing the CDs and not being reliant on the CD player at all... I've digitized my CD collection twice at this point. Once in the early 2000s to high bitrate MP3, then I redid the whole thing in the mid-2010s to change the whole library to FLAC so I had a bit-for-bit reproduction of the CDs and could back them up with no data loss. The computer I used is 10 years old, it takes it less than 5 minutes to read an entire CD and turn it into FLAC files. The software fills in all the titles, artist info, etc.. from the internet for you. It's really easy to do if you're working near the computer and just need to swap the CDs every 5 minutes as you do something else. I've still been buying CDs all along too, once you've done the first big batch it's trivial to do a few more here or there as you buy them.
And once you have the files as FLAC, ALAAC, whatever you are now free to use players/devices/setups that are better than any CD player and you have a path to play the higher definition stuff you can buy now instead of CD.