Luke and Jess arrive slightly late for the dinner party, with Luke implying he’s had trouble convincing Jess to come with him. While Lorelai assures Luke that it’s fine, Jess quietly slips into the Gilmores’ living room, which is off the hall. He looks at the photos on the mantelpiece, and touches the frame of one showing Rory wearing a pink cardigan, half picking it up.
Jess manages to check Rory out before he even meets her, and there’s no mistaking that he’s interested in what he sees. A boy who touches a girl’s photo is probably thinking about touching her, after all.
Lorelai finds him and surely has an idea what he’s up to. She smilingly bundles him out to the kitchen to meet Sookie and Jackson, but she must be getting suspicious of Jess. He’s crept around her living room without asking, and checked out her daughter’s photos. I imagine she is now telling herself to keep an eye on Jess, in case he has more creeping around to do.