RORY: Mr. Neville likes all things frilly.
RICHARD: Good God. He’s my broker.
Rory means that Mr. Neville likes wearing women’s underwear – although the mention of little girls wearing frilly underwear later in the episode gives this unfortunate implications.
It’s interesting that growing up in gossipy Stars Hollow has prepared Rory well for the intrigues of the country club world.