My own two stories of this from my Ph.D. work:
- I needed to make bioreductive metal complexes of a drug I was working on. Everybody in the literature had been using Co(III) and it was not working very well for various reasons. I thought, "hell, why cobalt? I'm just gonna try whatever metal salts I can find in the cupboard" because I knew next to nothing about inorganic chemistry. Lo and behold, copper (II) not only made beautiful crystals (that showed via crystal structure just how perfect copper was for this compound) but also even had the intended biological effect. It was only because I was like, "Huh. Why not try it," that I went from mediocre Ph.D. progress to actual interesting advance that solved a problem people in the area had thought intractable for about 5 years.
- One of the starting materials we needed for the compounds we worked on was a real pain in the ass to make. The reagents were nasty, the yield was bad, but the literature procedure was more straightforward than some and at least gave us SOMETHING. I toiled away at making it again and again for about two years, always having to make more because it would run out. Once I got an undergrad, I assigned him to churn out a few batches of this stuff. Whaddaya know but that guy gets 80% yield to my 30%! I was like, "DUDE what did you do that I did not do?" He couldn't find anything, we combed through his notebook, nothing. Finally I have him walk me through him setting up the reaction, he goes to add the catalyst, and uses a 5 ml syringe. Me: "Why are you using a 5 ml syringe to add 0.5 ml?" Him: "0.5 ml?......I was adding 5 ml......." This is why you should write EVERYTHING DOWNNNN especially when you first start and your detail muscles are not developed yet. Once we figured that out, everything in our lives went so much better, all because he screwed up and did it wrong in a good way (and I was able to extract out from him what the hell he did).
This is one of the things I love about chemical biLOLogy--you get to screw around with stuff that you know is based on molecules, and so it should make SOME sense, and it doesn't matter at all if you understand what you are doing at the time. You can learn it as you go along, and the chemist's molecularly mechanistic perspective helps you go back and dissect your dumb luck to find the places you pinned the right tail on the wrong donkey spleen (or vice versa).