Apparently I didn't heed the warning at the bottom of the configuration page for the NTP pool that mentioned a bug with RHEL's ntpd. My score suffered greatly when ntpd disappeared around midnight.

The SRPM linked in the bugzilla post wouldn't compile cleanly for me, so I'm now running a self-compiled ntpd. Hopefully that'll stay up and happy.