Welcome to the Age of Denial – NYTimes.com

Welcome to the Age of Denial – NYTimes.com.

Read this, it is important!

 

Watching Bacteria Evolve, With Predictable Results – NYTimes.com

Watching Bacteria Evolve, With Predictable Results – NYTimes.com.

This article, and especially the videos, are really cool. Carl Zimmer writes prolifically on evolution and I find his style very engaging. Especially his 2001 book “Evolution: the triumph of an idea” is a masterpiece. You should try to get the hard cover edition if you can find it, as it is a collector’s item. (It seems to be out of print though.)

Evolution is basically an optimization technique run by nature and it typically explores a space of solutions that is complex, multi-dimensional, and has many competing solution. (The “evolutionary landscape has many niches”.) So it is considered unlikely that starting from the same point evolution will end up with the same solution if an experiment is run multiple times.

In the specific case of these bacteria, however, at least under controlled lab conditions the scientists were able to recreate the same evolutionary path leading to “hyper-swarmers” repeatedly. This is not necessarily surprising nor is it in any way a contradiction. The key phrase is “controlled lab conditions”. In  nature conditions are not controlled and repeatable – all kinds of stuff is happening in unpredictable ways: meteors are crashing to the earth’s surface, volcanoes are erupting, CO2 levels are rising, solar activity is fluctuating. So evolution is bound to react to all these inputs and hence have most “chaotic” trajectories. (Chaotic in the technical sense of  unpredictable, not the colloquial sense of aimless or disorganized.)

 

 

 

Canadian study finds no correlation between MS and blocked neck veins – The Globe and Mail

Canadian study finds no correlation between MS and blocked neck veins – The Globe and Mail.

Again, rather than debating the actual result, I would like to focus on the methodology of this study as well as Dr. Zamboni’s original paper. Both actually used some from of AB testing, and both reported statistically significant result, but they are in direct contradiction to each other. How can this happen? Firstly, lets be clear, it CAN happen by sheer coincidence. But that has a low enough probability, so lets disregard that as a primary explantion. The most likely reason for the contradiction is either

1. outright fabrication (a possibility not that cannot be written off),
2. or more insidiously, a false premise or faulty setup of the experiment.

Hence the scientific approach places great stress on independent verification. And, in this case, Dr. Zamboni’s finding stands alone on one side while several independent studies, all in agreement that CCSVI has no significant correlation to MS, stand on the other.

It is important not to let this issue be clouded by anecdotal evidence from individual patients who were supposedly cured by Dr. Zamboni’s treatment. This is because a single case or a few cases cannot attest statistical significance. Also this type of evidence is highly selective (“selection bias”) – people reporting only positive results but not negative.

There is no alternative to AB experimentation, and even  then there are many pitfalls in experiment setup, so the results need to be reproduced in several independent AB-experiments. Getting to the truth is a difficult and laborious process – why should it be otherwise? Science is in many ways a very democratic field – but with a caveat. You get to vote only if you play by  the rules of the scientific method.

 

 

A Simple Device to Detect Concussions – NYTimes.com

A Simple Device to Detect Concussions – NYTimes.com.

Note the method used to validate if the simple puck-and-dowel test was reliable enough to detect concussions: a proper AB-experiment. Not sure if the sample size (25 players each in treatment and control) was large enough. But the approach is very scientific!

Also noteworthy is the problem of getting large enough sample sizes in practice. Its not just a question of paying people or using crowd sourcing. Here we actually need enough people in treatment group who have had real concussions! Now, I doubt if anyone would agree to get artificially concussed  for any kind of money, not to mention the unethical nature of such an experiment.

So there are some very real world obstacles in getting statistically significant results from experiments, that are not just related to laziness or lack of money.

 

An interesting chart from IEEE Spectrum

Graphs of industries/technologies showing exponential improvement in cost/unit. Which also translates fairly easily into units/volume etc, where relevant.

Noteworthy is the graph of DNA sequencing, where apparently
“Shotgun sequencing” is now considered an old  technology and the frontier has moved on.

Noteworthy by its absence is the graph of battery technologies. This has been one of the great disappointments in my opinion. Battery capacities and costs have essentially stagnated, and this has been a great drag on the possible proliferation of mobile computing. You can’t truly be mobile if you have to be always tethered to a power socket, or at least be near a power socket so as to recharge your numerous devices periodically (read: every day). I look forward to the day when I do not need to worry about charging batteries any more! It is a real pain in the ass …

‘Like’ This Article Online? Your Friends Will Probably Approve, Too, Scientists Say – NYTimes.com

‘Like’ This Article Online? Your Friends Will Probably Approve, Too, Scientists Say – NYTimes.com.

This is a great article that should raise alarm bells in the minds of all who do some kind of AB experimentation to understand user behavior and preferences, especially via crowd sourcing. The main problem as demonstrated in this article is the “herding effect” – users who see the preferences of other users tend to modify their opinion to go with the herd. An apparently this effect is very significant for positive opinions, and not so much for negative opinions. (Though I am a little sceptical of the later claim because from personal experience I have seen that negative opinions also tend to snowball.)

In any case, if you will recall the blog post I did a few days back about statistical significance, the importance of this herding effect is as follows: In AB testing, if possible keep the members of your control  (and of your treatment) isolated from each other. When this is impossible (for examples you cannot prevent patients from talking with each other about the effects of their treatments) take into account the statistical dependence of the subjects into the “null hypothesis” model. This basically means that the condition for statistical significance become more stringent and requires larger and more diverse sample sizes, which in turn increases the cost of the experiments.  But it is always better to do a few costly but correct experiments rather than doing many flawed experiments! There is no “law of large numbers” that can magically produce valid results from a plethora of faulty results. Just like in painting, if you throw all kinds of hues together, you end up with the color of mud!

 

 

 

 

Starting a new blog

Some personal stories, not technical. You may want to check it out.