(Rory opens the door and walks in. Emily looks in from the doorway.)
RORY: I know it’s looks small, but it’s really pretty. Come on. See, we had our bed right over there, and Mom put up this really pretty curtain around the tub so that it looked like a real bathroom. And we would just sit outside at night when the Inn would have parties, and we’d just listen to music and feed the ducks and . . . (Emily walks away) Grandma? Grandma wait, what’s the matter?
This is the potting shed next to the duck pond at the Independence Inn that Lorelai and Rory lived in when they first moved to Stars Hollow, as they had no money for accommodation (like the Holy Family, there was “no room at the inn”, and they were put in an outbuilding, so Baby Rory was just like Baby Jesus).
The shed is sturdy but rustic, and is stocked with gardening tools and plants, like any potting shed: it isn’t clear if those things were there while Lorelai and Rory lived there. Their bed is no longer there (they must have shared a single bed together), but the bath has been left, including the curtain that Lorelai put around it to serve as a bathroom wall. Lorelai mentioned that it has rosebud wallpaper, but the is shed painted white inside and doesn’t look as though it’s got the kind of walls that you could easily wallpaper.
It looks impractical for bringing up a baby, and we learn later that they moved to Stars Hollow in the autumn, so it would have been very cold as well (we don’t know what they used for heating). We don’t know how long they lived in this temporary accomodation, but long enough for Rory, who was only a baby when they came to Stars Hollow, to have some memories of it, and long enough that the weather became warm enough for them to sit outside at night. I would guess at least a year, and possibly two. Who looked after baby Rory while Lorelai was working is unknown.
This is the first time that Emily has ever seen the potting shed, and she is clearly distraught to discover the conditions her daughter and granddaughter lived in. Lorelai told Sookie that her parents visited them a few times at the inn while Rory was a baby, but they never saw where they slept at night. Lorelai was probably clever at keeping them away from the shed, but their lack of curiosity is surprising. Perhaps they were scared to push it in case Lorelai ran even further away.
In this case, it is Emily who runs away, too upset to spend any more with time with Rory or even say a proper goodbye to her. This incident serves as a device to keep Emily at a distance from Stars Hollow. Emily was having a good time with Rory, and was fitting in well with the townspeople, finding that she had things in common with Mrs. Kim and Michel. By showing her in the potting shed, it explains why Emily doesn’t visit Stars Hollow more often in the future.
Where they lived between the potting shed when Rory was a baby/toddler, and moving into their own house when Rory was eleven, is a complete mystery and never mentioned. Perhaps Lorelai saved up enough money to rent a cheap apartment for them, but renting would make it hard to save for a house. They could have lived in a friend’s house (with Sookie?), but if so, nobody ever refers to it.
In real life, it wouldn’t be legal for anyone to live in the potting shed under Connecticut zoning laws, but I’m not sure that would stop Lorelai anyway – rules were made for non-Gilmores!