I remember a joke from my school days; An eminent lecturer stands in front of a packed auditorium and places a small pink box on the bench at the front, he takes the lid off the box and inside there is a spider, the lecturer informs the gathering that this spider has been highly trained in order to demonstrate a new scientific principal, he then looks down at the spider and shouts "Spider, jump out of the box" and to everyone's surprise the spider jumps out of the box, the lecturer then shouts "Spider, jump into the box", applause spreads through the gathering as the spider obediently jumps back into the box. With a pair of tweezers the lecturer proceeds to pull each of the eight legs off the spider in the box, the spectators are intrigued and a little horrified that such treatment should be doled out to such an obliging little spider. The lecturer then shouts "Spider, jump out of the box", the spider just looks at the lecturer who again shouts "SPIDER JUMP OUT OF THE BOX" but the spider still remains in the box. Triumphantly the lecturer spreads his arms wide and pronounces to his audience that "I have now proven that when you remove a spider's legs you cause it to become profoundly deaf".
OK, not the best joke in the world and it certainly seems much more long-winded now that I see it written down compared to my first hearing of it in the playground however there is a core truth to be drawn from it. When conducting research you must make sure that you are asking the right question, that you have chosen the right method to ascertain a useful answer and that the conclusion derived from the results is valid. In the joke example above none of these conditions are met... that's what makes it (a little bit) funny.
My views on medical research are almost entirely informed from my reading of a most excellent book called 'Bad Science' by Ben Goldacre, it really is worth reading as it is well written, entertaining and hugely informative. Right at the start of the book he uses the phrase "the plural of anecdote is not data" which is relevant to us Vapers because there is so much anecdote and such a shortage of data about ecigs, furthermore what data there is is often generated from researchers funded by groups who have an interest in obtaining a result that serves their interests.
Over at the 'The Rest of the Story' blog Dr Michael Siegel highlights a study by Katrina Vickerman in which 2,500 people who called American State Quitlines were followed up 7 months later to assess the quit methods that they employed and their respective success rate. As Michael Siegal points out "instead of estimating cessation rates among a cohort of smokers who made quit attempts using these products, the study analyzed cessation rates of a large number of smokers who had previously tried to quit using e-cigarettes but failed, and then called a quitline because they had failed and wanted to try again" thus drawing any conclusions about the comparative effectiveness of ecigs as a cessation aid are invalid as a large proportion of those calling the Quitlines will have already failed to quit smoking using ecigs, if they had successfully quit by using ecigs then they would not be calling the Quitlines and would therefore not be included in the study.
Michael Siegal reports that the study authors conclude that "This study indicates that e-cigarette users were less likely than those who had never used to have quit tobacco at the time of the survey..." to which he responds "But the truth is that many of the electronic cigarette users in the study did not use electronic cigarettes in their quit attempts!".
It appears that the wrong conclusion is drawn by the study authors because the cohort of those questioned is skewed. The only conclusion that I can draw from these results is that Ecigs do not always lead to everyone who tries them quitting cigarettes, one might extend this view to postulate that ecigs are in fact so broadly effective that if even Ecigs won't stop you smoking then there is a good chance that nothing else will either, this is quite different to the way the authors of the report interpreted their results.
We should also ask if the question of the effectiveness of ecigs as a smoking cessation aid is valid in the first place. We vape as an alternative method of getting nicotine instead of smoking tobacco, we do so because it is far less expensive and that logic dictates that vaping ejuice is likely to be less harmful to one's health than the thousands of chemicals found in tobacco. Ecigs are not sold as tobacco cessation aids and it seems wrong headed to seek to critically evaluate them as cessation aids. It is also not clear to me that the researchers consider Vapers to be ex-smokers, it is not uncommon for people who should know better to conflate nicotine intake with smoking tobacco.
I would love to see a truly randomised, double blind, comparative study between ecigs and a cessation aid (e.g. nicotine patches) as opposed to the study discussed above which is certainly not randomised or double blinded. I would also like to see a large scale, long term study of a cohort made up of smokers with a view to extracting data of quit attempts, methods and success rates at fixed time periods and/or age and the long term health outcomes of the various choices made by the participants. These trials would need to be undertaken by an independent and disinterested body so as to avoid the appearance or temptation of bias. It would be very expensive to undertake these trials however I am convinced that smoking tobacco is such a danger to public health and so costly to the public purse that the cost is justifiable and worthy of governmental financial support even in these days of austerity.
I'd like to talk more about how I would like to see these studies constructed and undertaken so I may return to the subject in the future, for now though, should you wish to read further about this particular study you can see Dr Michael Siegel's post here, the extract of the study here and another report on the study at the 'Live Science' site here.