The use, abuse, and misuse of data by the U.S. military during the Vietnam War is a troubling lesson about the limitations of information as the world hurls toward the big-data era. The underlying data can be of poor quality. It can be biased. It can be misanalyzed or used misleadingly. And even more damning, data can fail to capture what it purports to quantify.
via Robert McNamara and the Dangers of Big Data at Ford and in the Vietnam War | MIT Technology Review.
During the class in the spring of 2012 he learned the graphics programming language CUDA, and that opened the doors for tweaking GPUs to divide advanced computations across the GPUs massively parallel architecture.
He knew he had something when he wrote an algorithm to connect millions of points on a map, joining the data together spatially. The performance of his GPU-based computations compared to the same operation done with CPU power on PostGIS, the GIS module for the open-source database PostgreSQL, was “mind-blowing,” he said.
via Fast Database Emerges from MIT Class, GPUs and Student’s Invention.
The files contain information on over 120,000 offshore entities — including shell corporations and legal structures known as trusts — involving people in over 170 countries. The leak amounts to 260 gigabytes of data, or 162 times larger than the U.S. State Department cables published by WikiLeaks in 2010.
via Massive data leak exposes offshore financial secrets – World – CBC News.
BigQuery is a cloud service that lets users analyze terabyte-sized data sets using SQL-like queries. It’s based on Google’s Dremel querying system, which can analyze data where it’s located (i.e., in the Google File System or BigTable) and which Google uses internally to analyze a variety of different data sets.
via Google BigQuery is now even bigger — Tech News and Analysis.
But the most influential new product may be the least flashy: a $100 million database built to chart the academic paths of public school students from kindergarten through high school.
via K-12 student database jazzes tech startups, spooks parents | Reuters.
If you want to see the future of Big Data, look no further than the nearest gaming-development studio. It isn’t all fun and first-person-shooting. Game developers are the sentinels of a variety of advanced IT techniques, placing them in front of the general IT population with regard to using real-time analytics and cloud computing, among other areas.
Can you make it such that—most calls are recorded—you can look at the calls after and figure out key things, see patterns via big data? And can you actually apply that?
via Xerox’s CEO Wants to Shake Up the Services Market | MIT Technology Review.
Intel says the distribution is optimized for the Intel Xeon processor platform. In its announcement, the company states it can analyze one terabyte of data, which would previously take more than four hours to fully process, can now be done in seven minutes.
It also gives other data. According to the findings, Latin Americans tended to answer their calls significantly quicker than the rest of the world. In Panama, the average time the phone is left ringing before an answer is 6.42 seconds, while in Libya it’s almost double that – 11.16 seconds.
via Start-Up Tawkon Rides Radiation Scare To Build Phone Coverage Maps.
Data is the new currency.
“So think of a host beaconing out to a C2 (command-and-control) site on a regularly scheduled basis,” he tells Dark Reading. “If an analyst can isolate the suspect host, they can eyeball a graph to see that they’re reaching out to this host regularly. But with a big data approach, you can create a rule that computes and analyzes the interval between sessions and determines whether we’re talking about normal human activity, or machine-generated — which is innocuous — or scheduled activity like malware might do.”
via RSA, IBM Bet On Big Data Analytics To Boost Security – Dark Reading.
I recently caught a piece of malware on a PC on my open wifi doing something similar.