Test Emissions Where Cars Pollute: On the Road – The New York Times

In the wake of the Volkswagen scandal, we should remotely monitor vehicles’ emissions. Some states already do.

Source: Test Emissions Where Cars Pollute: On the Road – The New York Times

Need I say more?

Ok, I will. 🙂

It is also noteworthy that the telemetry being proposed here is coming not from the car itself but from the roads on which that cars drive – i.e. remote sensing from the environment of the product. This can be useful in cases like monitoring pollution, gross usage and security. But there are other use cases like understanding user pain points in the product, when environmental measurement may not be sufficient and we do need to have actual in-product telemetry. For example, perhaps fender-benders and swerving can be recorded by an outside sensor, but to record hard brakes or sudden high revving on engine we probably need to instrument the car itself. Another problem with environmental telemetry can be about how to identify the car uniquely. Perhaps via a photo of the license plate? But people move licence plates between cars, and a single car can also have multiple license plates. So there is a problem of identification and disambiguation. In contrast, in-app telemetry can simply upload the car’s VIN number (which is hard coded in the car’s firmware and stamped on the chassis) and that is something that stays with the car forever!

A Math Problem From Singapore Goes Viral: When Is Cheryl’s Birthday? – NYTimes.com

A Math Problem From Singapore Goes Viral: When Is Cheryl’s Birthday? – NYTimes.com.

A nice problem to hone your probabilistic/logical reasoning. The trick is to see every piece of data for what it is and how it adds to previous knowledge. Also important to keep in mind at every step three types of knowledge:

1. What Albert knows
2. What Bernard knows
3. What is common knowledge (i.e what you, the reader, knows.)

Happy reasoning!

PS: It is possible to produce probabilistic variants of this problem by tweaking the dates so that certain uniqueness properties are not satisfied. In that case the riddle could be posed as: What single binary question can you ask Bernard to resolve the birthday?

 

Vaccine Critics Turn Defensive Over Measles – NYTimes.com

The anti-vaccine movement can largely be traced to a 1998 report in a medical journal that suggested a link between vaccines and autism but was later proved fraudulent and retracted. Today, the waves of parents who shun vaccines include some who still believe in the link and some, like the Amish, who have religious objections to vaccines. Then there is a particular subculture of largely wealthy and well-educated families, many living in palmy enclaves around Los Angeles and San Francisco, who are trying to carve out “all-natural” lives for their children.

via Vaccine Critics Turn Defensive Over Measles – NYTimes.com.

 

I already blogged about the scientific method earlier today and vented my ire about charlatans and frauds who bamboozle the public with pseudo science. Here is another example – with consequences that are not merely inconvenient but deadly.

The overwhelming scientific consensus is that vaccinations do not have clear links to autism. Comprehensive vaccinations have had near miraculous results in third world countries like India where infant mortality is falling rapidly. The positive effects of vaccination are clear and indisputable.  And yet educated and seemingly liberal minded parents put their children (and the children of others)  at risk by not vaccinating as recommended. How can we explain that?

Only one answer comes to mind: A lack of science education, in particular education about the scientific method – theory making, experiments, statistics and confidence levels. Perhaps we need to introduce these ideas as early as kindergarten!

Speck of Interstellar Dust Obscures Glimpse of Big Bang – NYTimes.com

Now a new analysis, undertaken jointly by the Bicep group and the Planck group, has confirmed that the Bicep signal was mostly, if not all, stardust, and that there is no convincing evidence of the gravitational waves. No evidence of inflation.

via Speck of Interstellar Dust Obscures Glimpse of Big Bang – NYTimes.com.

I cite this article not so much because of the subject matter (the theory of inflation in cosmology and the idea of gravitational waves, which  are fascinating topics in their own right), but more so to emphasize and applaud how the scientific method works.

Several things are noteworthy. Firstly, the Bicep group was the one which did the original work that suggested convincing evidence of gravitational radiation based on polarization patterns of the CMBR.  Even in their original work they did mention the possibility that the patterns they were detecting may have come from near-field optical effects of interstellar dust, but they were confident that they had compensated for the amount of dust expected.

Secondly, the overall cosmology community did not just accept their claims at face value. They looked at it critically. In particular they asked – did you really account for all the interstellar dust present? Perhaps you made an underestimate? But this was respectful scientific criticism, not vilification.

Thirdly, they backed up their criticism by putting out evidence from a very different and unrelated source of measurement – the Planck experiment for CMBR measurement in various parts of the sky.

Fourthly, the Bicep folks heard out these criticism patiently and in fact works with their critics from ESA to integrate the Planck measurements into their own data and recalculate their own estimates.

Fifthly, after doing this joint analysis they realized and publicly accepted that while there was still a distinct “signal” left over after compensating for interstellar dust, it was not large enough to conclusively prove (remember my post about 95% confidence?) the existence of gravitational waves and inflation.

Finally, cosmologist have not utterly rejected the idea – absence of proof is not proof of absence! Many teams will continue to work on this since inflation is an elegant idea that explains a lot of things about our universe that would otherwise be hard to understand. Perhaps more sensitive measurements may yet resurrect and buttress the original Bicep claim. All we can say if that at this point in time we cannot say so with confidence. Which is a a fair enough statement to make. The universe has been around for billions of year, so a few years’ delay in our understanding  of its origins can hardly cause any harm (except perhaps delay the career advancements of some scientists. And even that should not happen in principle –  a negative finding is still a valid finding, and the finders should be given due academic recognition for that.)

 

I am truly in awe of the scientific process, and that it works at all when the rest of our world is so screwed up. Scientists are humans after all, and I am at once humbled and elevated to see imperfect being coming together to build something far more perfect and pristine than themselves.

It is noteworthy that a similar process and consensus also exists in most other scientific disciplines – prime examples being evolution and climate change. There is no real dispute among the vast majority of scientists about the validity of these theories – they are well past the 95% confidence threshold. It is only political charlatans and religious demagogues  who want to muddy the waters by raising unfounded criticism and making ludicrous counter claims. They present no compelling evidence, they have no respect for the scientific spirit or method – all they care about is furthering their own narrow political or religious agenda. And in doing so they hold the whole world hostage – putting our children’s future in peril.  Where is the justice in that?

 

 

 

 

 

 

Microsoft Azure Machine Learning combines power of comprehensive machine learning with benefits of cloud – The Official Microsoft Blog – Site Home – TechNet Blogs

Microsoft Azure Machine Learning combines power of comprehensive machine learning with benefits of cloud – The Official Microsoft Blog – Site Home – TechNet Blogs.

The United States of Metrics – NYTimes.com

The United States of Metrics – NYTimes.com.

This is a great article. The author makes several great points. And even though I don’t agree with many things he says, it is undeniable that he is bringing up questions that are of deep significance.

Should we measure everything we can? Should we optimize everything we can? This is essentially the main quandary.

You can easily guess where I stand on this issue (see the “about” section of my blog.)  But I also worry deeply about how the data revolution can create a new kind of division in society – between people who are comfortable and savvy about data and statistics, and those who are not. (I would refer you to my very first blog post from two years ago.) And even if most people do jump aboard the data revolution and teach themselves to be comfortable with ubiquitous metrics, will that take away some of the joy of living?

When making love should we be thinking of our “stats”? When standing on the edge of the grand canyon should we be snaps photos and posting them on Facebook, or simply breathing in the beauty of the surroundings and contemplating the grandeur of nature? When eating a great steak should we be counting calories, or rather savoring every juicy morsel?

And what about those “messy” things in life that the author refers to like birth and death, sickness, feuds, divorce, professional challenges, friendships made and broken? All those things that make us “human”? Can they be improved or optimized by metrics? (I really don’t know the answer).

What we need is a human face to data sciences and metrics. A way to consume data and optimize our lives without thinking like an actuary all the time. In many ways this is a great, and perhaps solvable, engineering challenge. I have every hope that human ingenuity (that most difficult-to-measure thing of all!) will in fact prevail and provide an egalitarian solution.

 

Probability and Belief

I recently had an interesting conversation that inspired today’s post. What I want to talk about today is the interpretive and philosophical aspect of probability theory. And I promise I will keep it a light hearted affair, nothing too heavy. Also a disclaimer at the outset: the word “belief” in the title only refers to rational belief , and hence should not be construed as also referring to religious or political beliefs.

As I have talked out probability and information theoretic quantities, I have always stated or implied a “frequency of occurrence” interpretation of probability. For example I said “if we see a hundred examples of commuters on a bus” then we can learn something about their probability distribution and then perhaps make predictions about their sometimes hidden attributes.

But this is not the only way to think of probability. The theory applies more generally as a method of rational manipulation of beliefs and can be applied to questions or experiments that in principle cannot be repeated. For example, consider the currently urgent question “will the Euro survive the economic crisis in Europe?”. You can very well give an answer that “the probability that it will survive is 80 percent”. Of course there is no “frequency” aspect here, the Euro experiment happened once and will never be repeated again in its current form. Of course you could have been influenced by examples of similar attempts at monetary union elsewhere. But nevertheless those are not exact replica experiments so the frequency interpretation does not apply.

So what is going on here? You are just expressing a belief about the Euro’s viability. Now unlike cable TV pundits or politicians, hopefully you are expressing that belief as a result of rational thinking. In fact ideally you should have started from some atomic postulate beliefs which most of us would easily agree with and then work your way upwards in a consistent manner till you could answer the “top level” question about the prognosis for the Euro.

The basic point I want to drive home is that probability theory offers us a (and in my opinion the only) method for rational manipulation of atomic beliefs to answer high level questions. This essentially boils down to a repeated and consistent application of Baye’s law and conditional probability. In this context probability is interpreted as an expression of confidence. Also, if the theory answers that none of the possible answers has a high confidence value ( meaning that the entropy of my belief is high) then that is not a “wrong answer”. It is in fact telling us that there is no rational basis for making a judgment call on that question, and hence we should reserve judgment. This last aspect of probability theory is the most interesting for me. There are innumerable worldly questions that have no “good answer”, meaning basic postulates do not lead to confidence in any option. I think in such a case we should just shut up and not waste our breath!

However that is not the end of the matter. Given basic postulates , there is still the delicate question of choosing the right probability model before we can begin rational manipulation towards answering top level questions. But since we do not have hundreds of examples to “learn” from, how do we select and tune this model? This in my opinion is the most important philosophical question of all time. And I, personally, have tentatively found a partial answer: I will first try those models that are the “most beautiful”.

Now you may frown at this statement, but bear with me. Consider the recent example of particle physics experiments confirming the existence of a Higgs like particle. Now the theory that predicted the existence of this boson was in place decades earlier and was a purely abstract construct. Physicists used some base postulates about how particle physics ought to work (which had some experimental evidence) and then from among the countless possible theories zoomed in on the mechanism of the Higgs Boson as a possible model to explain how elementary particles could acquire mass. I would argue that this choice was driven in no small measure by notions of “beauty and elegance”. There are many other ways you can explain mass. But none is as elegant as the Higgs field.

So there it is. Many posts ago I hinted that I would connect up notions of probability and beauty and show that they fit in a bigger scheme. This is that scheme: probability theory can be used to talk about and manipulate beliefs. But to do so we need to select a good model. An artists notions of “sparsity, beauty, and elegance” are exactly the principle that we need to use in order to select good models. So ultimately our notions of belief, and even justice, are formed inexorably from our aesthetics. And hence the study of fine and liberal arts is not an effete affliction, but rather forms the foundation on which the entire intellectual development of our society depends!

The invasion of the Computer Scientists

This is a small  post today, done while I wait , as usual, for my wife to finish shopping.

My question for today is the following : are computer scientists taking over the world ? Now don’t be mistaken , I do not mean this in a menacing apocalyptic way . What I mean to ask is , is being a computer scientist becoming a prerequisite for many other professions such as biologists, astronomers, physicists , city planners , doctors , lawyers and even poticians. I can see you raising your eyebrows and muttering to yourself “what Koolaid has he been drinking today?”

But believe me, I do not make this statement lightly . I can give you documentary evidence how some of the biggest developments and advances in these professions have come from the ability to collate vast amounts of data and tease out meaning and intelligence from it by using some pretty arcane algorithms.

Just as two examples consider that:

Big law cases are now argued in court by teams of lawyers using some pretty sophisticated analysis of precedents, as well as statistical predictions about judges and juries based of archival case files .

The project for discovery of earth like planets in deep space has made great progress by looking at vast amounts of observations of the night sky and trying to figure out all kinds of tell tale signs of extra-solar planets, like wobbling or blinking stars . This kind of astronomy was simply impossible before the advent of big data capabilities .

So here is my thought for the day :

To be good at many interesting professions in today’s world you have to first and foremost be a good data / computer scientist. With that skill in hand you can pretty much go into any field and in short order pick up enough domain knowledge to start making significant contributions. This is kind of an inversion from the days of old when people had to first be very good at their domain , and then some facility with computing technology would be an additional advantage . Today big computing skills are  not simply an asset , they are a core requirement !

I do not mean to belittle other professions (in fact I myself am not a computer scientist by training ). I just want to make the point that the nature of modern professions is changing rapidly, especially in science and engineering , and we all need to adapt. In a sense all modern professions have become very empirical (Aristotle would be chuckling in his grave!), and data driven reasoning and decision making is now becoming the norm, with less emphasis on hunches, guesses and blind luck.