Me too. Once in a Blue Moon, I've even been known to buy something twice. Once subbed, and then again when the dubbed version is released. Conversely; I've purchased anime, and then discovered a more complete set. These cases are rare, but do happen when regarding an anime I really like.
I don't do it willy-nilly though. The Madoka L.E.s for example were out of my league. There was an Anime Classics Complete Series Edition of Gunslinger Girl that I wanted too, but I couldn't justify it considering I already own everything contained in it piecemeal.
No matter where you go, there you are.