Rory walks away, not realizing her bracelet had fallen off while she was on the bridge (the bracelet Dean gave her as a sixteenth birthday present). Jess picks it up and puts it in his pocket – just like the “little boy” who picked up the lost love letter in the nursery rhyme, A-Tisket, A-Tasket.
It is unclear why Jess keeps Rory’s bracelet, but most likely just the sentimental pleasure of having something of Rory’s he can hold and touch. He probably intended to give it back at some point, the same way he returned her book. He may have also thought the handing back ceremony would be a flirtatiously teasing one – “Oh, you were looking for this? Another one of my magic tricks“.