I just happened across a musing on the necessary materials for creativity that exactly mirrors what I've observed myself:
"The first law of creativity: The act of imagination depends directly on the richness and variety of a person's previous experience because this experience provides the material from which the products of creativity are constructed. The richer a person's experience, the richer is the material his imagination has access to."
This is from the writings of Russian psychologist Lev Vygotsky, discussed in a post HERE at Keith Sawyer's blog Creativity & Innovation.
Mind you, a huge heap of building materials is not, in itself, enough to imagine building a creative house... The desire to build, the energy and time to build, and curiosity and wonder are even more important. Only look at the vast numbers of richly experienced people who have never used that experience creatively.