This is the song that the town troubadour is singing when Rory gets off the bus and is met by Dean. It is the first appearance of the troubadour in the show, played by indie rock singer Grant Lee Phillips, and named Grant in the credits.
It is ambiguous whether the troubadour is actually meant to be the real Grant Lee Phillips who (like many other other celebrities) also exists in the Gilmore Girls universe, or whether Phillips is playing a fictional character who just happens to have the same first name. Perhaps he is a sort of parallel universe version of himself.
Sunday Best was a bonus track on the Australian release of Phillips’ 2001 album Mobilize, however that was months after this episode aired.