Here is one source that defines your concern about "holes" or defects in the brain caused by smoking marijuana.
http://www.mfiles.org/Marijuana/marijauna_brochurea02.pdf
(This document requires Adobe Acrobat to read)
It states (for those who don't want to read):
------------------------------------------------------
MYTH: Smoking marijuana has no damaging effects on the brain.
FACT: Recent advances in brain imaging, such as SPECT (single photon emission computed topography) scans, now allow doctors to see the functional aspects of brain activity. It can detect disorders such as ADA (attention deficit hyperactivity disorder), depression, OCD (obsessive
compulsive disorder) and ODD (oppositional defiant behavior disorder. This groundbreaking technology also dramatically reveals the damaging affects of various drugs, such Brain with normal blood flow 12 years of marijuana as marijuana, by detecting various medical and neuro-behavioral disorders. The column at far left shows SPECT images (top-down surface view) depicting a normal brain and a brain affected by chronic marijuana use. The images below show the underside surface SPECT images where defects, or holes, appear in areas of decreased blood flow and brain activity. Defects of this type have been associated with attention problems, disorganization, procrastination and lack of motivation.
--------------------------------------------------------
This document has pictures comparing a normal brain with two pictures, one that says "Marijuana affected brain" and another that says "12 years of marijuana." The differences are pretty startling.
Besides this source, I found no others. However, an interesting note that I found while searching for documents pertaining to this is that chocolate has a few chemicals that react to the brain in the same way that THC (the main chemical in marijuana) does, but that is another story.
After personal experience and a bit of research, I can only say that smoking pot once every two weeks to a month is safe, but smoking everyday, even once a week, can be hazardous. Marijuana can lead to cancer, respiratory difficulties, amongst other things. It does help the human body relax a lot, even if the emotions are a bit over-shot.