... is the root of all evil, according to either Dijkstra, Knuth or Hoare. Whoever it was, I figure that this is good company to be in. (Assuming that the original statement was a product of personal experience).
Unfortunately, premature optimisation is not an easy habit to give up. You might as well say that unnecessarily general class hierarchies are the root of all evil (another of my recent problems).
In a combination of both problems, I spent most of yesterday reversing the rendering order of my basic layer abstraction. The first implementation had rendered each layer in turn, followed by those above it. There is no point in recalculating a layer that hasn't changed, so unchanged regions from layers above are preserved in the local cache. Of course (?), this is completely the wrong way around. It is the layer on top that should be the main priority for rendering, because this is where the user is currently looking/interacting. It is the layers underneath that should be cached, because they are unlikely to change.
So failure to take a user-centric view in the original architecture meant that I had to completely reverse the caching order (not yet finished, in fact). If I hadn't spent so much time implementing the caching in the first place, reversing the rendering stack would have been a one hour job instead of a two day one. A clear case of premature something or other.