JESS: Hey, the girls that I like don’t give a damn about me! And unlike some other people I know, I’m not gonna sit around hoping that they change their minds and suddenly notice me.
LUKE: What’s that supposed to mean?
JESS: You fixed any neighbor’s porches lately? Or you go on a picnic or you get rooked into giving a ridiculous speech at a high school?
Jess tells Luke that his cynical view of relationships has partly been formed by watching Luke pine for Lorelai. He seems to have become indignant at the way Lorelai talks Luke into things, and the way Luke offers her free handyman work around the house. Jess seems to have decided that whatever else he might be, he’s not some sap who’s going to make a fool of himself over a girl.
Jess mentions Luke fixing Lorelai’s porch, which happened before Jess moved to Stars Hollow. Either he’s fixed it again recently, or someone told Jess about Luke fixing the porch. I presume that it was Luke himself, because Rory didn’t seem to know about it – it was early in the morning, and she blamed Lorelai for the noise that Luke made, before going back to bed. I suppose Rory might have accepted Lorelai’s explanation afterwards, and later told Jess about Luke fixing the porch – she may have thought it was a funny story.
The Annotated Gilmore Girls 