How to improve MPlayer video quality by selecting a "better" video rendering/filter?
You can choose a different video filter.
I use hqdn3d which gives me the best quality. Be careful on machines with low CPUs.
To use hqdn3d, put this line into your ~/.mplayer/config:
Check more options of video filters in MPlayer manual.