“There’s really one significant missing piece to Libby’s story. How did she get from A to B — from Desmond to the mental institution? We know the answer to that question, but the only way to tell that story is through another character’s flashback, and that character would have to be another character on the show who is not among the beach dwellers.” – Damon Lindelof.
My thought is that this character could be Penny. And that Libby ends up in the mental institute after having a break down when Desmond goes missing in the boat that she gave him. If that alone doesn’t tip her over the edge then maybe Penny seeks out Libby and blames her for Desmond’s disappearance. Hopefully this is something that we will find out in season 4, which Carlton Cuse has hinted at.