Isaac Asimov's essays have been favorites of mine since I was a teenager, and while I can't claim to have read them all (he was the most prolific writer in the history of the world, if one excludes 'writers' who have computers write for them) I have read a lot. So I was excited to hear that a previously unpublished essay of his, On Creativity. And like many of his essays, this is spot on.
To summarize his conclusions, intellectual creativity (creation of startlingly new scientific ideas in particular, but not only that) tends to occur when previously unconnected ideas are examined together by a person in a conducive situation. And, he argues, a key feature of that conducive environment is the freedom to be playful, to unabashedly look foolish, to pursue ideas that don't seem likely to go anywhere with people whose expertise has no obvious connection to one's own. He implies, and it is at least as true now as when he wrote it in 1959, that the structure and strictures of science-as-a-business (including in academia) tend to discourage this. Connecting previously unconnected ideas is less likely when everyone is a specialist in her own field, not only unaware of the big ideas in other areas of science, but obligated by the strictures of specialist journals, specialist department, etc. to not wander too far afield. In the world of reputation building and publish or perish, things like playfulness, acceptance of foolishness, and exploration of uncertain goals is potentially fatal. Funding applications not only require that you know exactly where you will end up, but also that you already have a significant portion of the data needed to get there.
At previous jobs, and in previous stages of my life, I often felt (and was told) that my intellectual creativity was my greatest strength. As things now stand, I have surprisingly little space for creativity, and when I do come out with something really original, I get something along the lines of, "Huh. That's different. What about this other thing that we all know about?" So the question I must ask myself is, how (and where) can I find a place where my creativity is an asset, not only for me, but for science and the world?