Hacking into the Indian Education System

Technically put, I merely needed to write a script to iterate through the various school IDs, check the different servers, and start with a student ID of 1 yet have a way to detect when there were no more students for a given school. I had to retrieve the resultant html files and parse them to extract all the useful information – Name, Date of Birth, ID, School, Marks.

via Hacking into the Indian Education System – On the Stepping Stone – Quora.

Several hours later, I had all the ISC and ICSE results on my very own computer, in a bunch of comma-separated value files. It was truly incredible. 26 megabytes of pure, magnificent data. An Excel file I couldn’t scroll to the bottom of. Just for kicks, I Ctrl+F’d a few names I knew and what do you know? There they were. Line after line of names, subjects and numbers. It was truly mesmerizing.

BSA Study Demonstrates Open Source’s Economic Advantage

So what this all boils down to is that the fundamental premise of the latest BSA study – that licensed proprietary software is better in many ways than pirated copies – actually applies to open source software even more strongly, with the added virtues that the software is free to try, to use and to modify. That means the potential economic impact of free software is also even greater than that offered by both licensed and unlicensed proprietary software. It’s yet another reason for governments around the world to promote the use of open source in their countries by everyone at every level.

via BSA Study Demonstrates Open Source’s Economic Advantage – Open Enterprise.

Survivorship Bias

Taking survivorship bias into account, Wald went ahead and worked out how much damage each individual part of an airplane could take before it was destroyed – engine, ailerons, pilot, stabilizers, etc. – and then through a tangle of complicated equations he showed the commanders how likely it was that the average plane would get shot in those places in any given bombing run depending on the amount of resistance it faced. Those calculations are still in use today.

via Survivorship Bias « You Are Not So Smart.

Simply put, survivorship bias is your tendency to focus on survivors instead of whatever you would call a non-survivor depending on the situation. Sometimes that means you tend to focus on the living instead of the dead, or on winners instead of losers, or on successes instead of failures. In Wald’s problem, the military focused on the planes that made it home and almost made a terrible decision because they ignored the ones that got shot down.

NASA’s system for avoiding collisions with space junk

Potential collisions are flagged for monitoring if there’s simply a high probability of conjunction. Typically, the probability goes down after a couple of additional days of tracking, but in rare cases this doesn’t happen (and, in a few, the probability went up with further monitoring). When the probability doesn’t go down, the software can calculate a maneuver that will reduce the probability of collision to an acceptable level. The solution will take into account other potential hazards as well as mission requirements—some Earth-monitoring satellites can’t orbit above a certain altitude and still perform their jobs.

via Saving Fermi: NASA’s system for avoiding collisions with space junk | Ars Technica.

Reasons to believe

It’s time to assume the mantle of Defender of the Faith. I’m going to give you ten arguments for believing P!=NP: arguments that are pretty much obvious to those who have thought seriously about the question, but that (with few exceptions) seem never to be laid out explicitly for those who haven’t. You’re welcome to believe P=NP if you choose. My job is to make you understand the conceptual price you have to pay for that belief.

via Shtetl-Optimized » Blog Archive » Reasons to believe.

How Facebook Built Natural Language into Graph Search

The engineers used a weighted context-free grammar (WCFG) to represent Graph Search’s query language. Think of a tree, with the root or base as the “Start” of a particular query. Facebook calls this the “parse tree,” and the various “limbs” branching from the root include verbs, objects, etc. The “leaves” at the top are the terminal symbols, or entities such as users, cities, employers, groups, and the phrases that link those entities together. It’s perhaps easier to diagram than explain:

via How Facebook Built Natural Language into Graph Search.

Wolfram Alpha Drills Deep into Facebook Data

At this year’s South by Southwest (SXSW) conference in Austin, Texas, Wolfram Alpha creator Stephen Wolfram offered up some interesting details about his computational engine. Wolfram Alpha contains more than 10 trillion pieces of data cultivated from primary sources, along with tens of thousands of algorithms and equations. Solving complex math problems is one of the system’s key abilities.

via Wolfram Alpha Drills Deep into Facebook Data.

More information from Data Science of the Facebook World

Some of this is rather depressingly stereotypical. And most of it isn’t terribly surprising to anyone who’s known a reasonable diversity of people of different ages. But what to me is remarkable is how we can see everything laid out in such quantitative detail in the pictures above—kind of a signature of people’s thinking as they go through life.