When you last used your supermarket loyalty card, or pulled up just as the traffic lights turned red, you probably weren’t thinking much about math. Why would you? Most of us see mathematics as baffling school algebra that has no application in real life. But you probably meet hundreds of bits of complex math every day without realizing it: the discounts on your loyalty card are the result of deep data analysis, and those traffic signals run on an algorithm that determines when the lights will switch to green.
In fact, large bits of modern lives are secretly underpinned by complex algorithms — the mathematical equivalent of cookery recipes, which take a piece of information and turn it into an action or decision. Algorithms are like computer programs or flowcharts — a sequence of steps that examines what is happening and comes to a conclusion. Take traffic lights: The computer controlling them asks a series of related questions. What time of day is it? When did the lights last go red? Has a pedestrian pushed the button at the crossing? The algorithm guides the computer, step by step, to change the lights.
Algorithms are pervasive, even controlling, in our lives. Book a low-cost flight and an algorithm will determine how much the tickets will cost, depending on supply and demand; arrive at the airport and air-traffic algorithms will determine which place in the queue your plane gets. Watch the weather forecast on TV and the predictions will have been fine-tuned by an algorithm; listen to the radio and the playlist may have been generated by one. Supermarkets, in particular, are a hotbed of algorithms. Almost every aspect of their operation — from deciding the order that products are stacked on the shelves to picking which special offers to run — is determined by a computer.
And since computers are increasingly dominant in our lives, algorithms are increasingly important — and nowhere is this more apparent than on the internet. In the online world, mathematical analysis isn’t just important: the algorithm is king. Everywhere you turn online, companies are using algorithms in their quest for success. From Google’s search results and Apple’s music recommendations to Amazon telling you that “customers who bought this item also bought...” algorithms are at work.
“There is no way, with the size of the internet, that one can do exhaustive searches,” says Marcus du Sautoy, professor for the public understanding of science at Oxford University, and one of Britain’s leading mathematicians. “So you have to rely on mathematics to give you clever and fast ways to get information.”
The British online supermarket Ocado, for instance, analyzes every detail of every activity — from the choices shoppers make to warehouse movements — to make sure there is enough stock in the right places, and to help the company plan future strategies.
“We’re using complicated forecasting algorithms to predict demand,” says Jon Rudoe, head of retail. “The customer sees nothing. It all looks very simple, but it’s actually governed by complex mathematics in the background” — mathematics we can put to use because we now have technology that didn’t exist a decade ago.
Since these recipes have helped internet companies cook up billions in profits, the precise details of the mathematical mechanisms are jealously guarded as among the companies’ most valuable assets. Forget the recipe for Coke or the Colonel’s blend of herbs and spices, these are the trade secrets of the 21st century. And wherever there are secrets, there are people desperate to unlock them. Around the world, countless hours and millions — perhaps billions — of dollars are spent trying to unravel the inner workings of the web’s most powerful algorithms.
Among those searching for answers is Russell Davies, an advertising consultant from London. His fascination started when he decided to try to understand how to lift his book — a guide to Britain’s greasy-spoon cafes called Eggs, Bacon, Chips and Beans — up Amazon’s rankings. On the surface, the Amazon charts seem straightforward. They are largely based on sales, but there is also analysis of buying activity so that a sudden rush of orders can push a book, CD or DVD up the chart. The rewards for chart-topping are enormous: it can drive thousands more sales. But there are other benefits too, including (crucially) a shot at appearing in the algorithmically generated “customers who bought this also bought...” section, which is hugely popular. Davies thought he could try a variety of tricks to shift his book up the bestseller list and — if he succeeded — work out the secret ingredients behind Amazon’s recipe.
“I started to think about what makes the number change and realized that it’s a big secret, which made it more interesting,” he says.
Over the course of several months it became a minor obsession; after all, for writers and publishers, even a small insight into Amazon’s algorithm could be like discovering the Rosetta Stone.
“The Amazon ranking is the only feedback the average author gets on how their book is doing, and you’re desperate for feedback,” Davies says.
He wrote about the book in newspapers and online; he asked friends to buy copies and even bought a few himself, all the time closely monitoring whether his actions were reflected by his place in the rankings. But although he had a few moments when he seemed close to a breakthrough, he hasn’t cracked the code.
“I learned that you can’t really influence it,” he concludes. “Thinking about it, it’s not that surprising — you’re in such a massive pool of data that a few sales here or there are just invisible.”
Amazon is not alone in working hard to keep its methods confidential. EBay’s reputation system is largely based on user feedback, but the company constantly works on adaptations aimed at stopping scammers from getting status they don’t deserve. Apple, similarly, has mathematical secrets that are increasingly important to its business, among them a recent addition to the iPod, the new “Genius” function that creates playlists of similar songs.
On the surface, Genius looks like hi-tech wizardry. It takes a song you own and works out similar music that you might like to hear: Stick in a shiny happy song and you’ll get 25 tracks of sunshine back; give it a mournful dirge and you’ll end up with a mixtape to die to. Underneath all the whizziness, though, the Genius function is really about number crunching. Apple analyzes the song choices of millions of other iTunes users around the world and, based on this, is able to take a stab at which songs match your seed track. And it’s profitable, too: Genius can encourage users to hear and buy songs from iTunes that they knew nothing about.
Another aspect of Apple’s business it keeps quiet about is the way its iTunes charts are put together. While the charts appear largely based on sales and the number of times people listen, iTunes is also believed to use a so-called “decay algorithm” to give more weight to very recent activity. The company refuses to confirm how the system works, although little pieces of information have slipped out in the past. Its podcast chart, apparently, is “driven purely by an algorithm that looks at new subscriptions during the past week,” for example.
Nobbling the mathematics behind iTunes is almost impossible, but some people have learned how to subvert it the old-fashioned way. Little-known group Hit Masters rode high in the charts earlier this year with a karaoke cover of the recent Kid Rock hit All Summer Long. They managed a modern twist on the age-old tactic of riding somebody else’s success, when they noticed that the original version was not available to buy on the US iTunes store. Their speedily produced cover appeared high in the search for the Kid Rock song, making them more likely to get downloaded; more downloads meant a higher placing in the chart, which itself generated more sales. It’s a self-fulfilling cycle that eventually pushed Hit Masters to a chart high of 19 in the US Billboard Hot 100.
But the efforts of Amazon, eBay and Apple pale in comparison to the most famous algorithm on the Web: Google’s algorithm is the mathematical engine that drives the web’s most powerful company.
“It’s not that Google is smarter at the maths, it just had a better recipe,” says Danny Sullivan, editor of the SearchEngineLand.com Web site. “It had ingredients that other people didn’t.”
Given Google’s dominance of web search, getting a high ranking there is a guarantee of more clicks — and more clicks means more money. With so much at stake, decoding Google’s algorithm has become an industry in itself. Experts in search engine optimization know the basics of Google’s operations — they are well documented — but the California company now spends vast amounts of time and energy trying to keep its formula secret.
“Large sites such as Amazon and Google are tweaking the internals of their systems almost constantly,” says Jon Kleinberg, a professor of computer science at Cornell University in New York state. “The front end of Google looks the same to us, but behind the scenes they can be busily swapping out ideas.”
Kleinberg believes the internet is making so much data available about how we relate to each other that we could soon be able to create algorithms for the social interactions we always thought were too complicated for math, such as controversy, disagreement or fame.
“They’ve always been fleeting, ephemeral, invisible and essentially unknowable,” he says. “Now we can try to get in there and understand why, at a microscopic level.”
All of which will have potential applications, both corporate and individual. Your PC could use algorithms to recognize exactly what document you are looking for, or to predict which news stories you might be interested in when you log on. Your mobile phone could recognize you are in a bad mood and screen your calls automatically, allowing only people that it has determined are your closest friends to get through.
Learning to sift through the vast amount of information being sent across the internet every second to divine people’s feelings or intentions could, Kleinberg believes, be the next great technological leap. Mathematicians rule!
As Taiwan’s domestic political crisis deepens, the opposition Chinese Nationalist Party (KMT) and Taiwan People’s Party (TPP) have proposed gutting the country’s national spending, with steep cuts to the critical foreign and defense ministries. While the blue-white coalition alleges that it is merely responding to voters’ concerns about corruption and mismanagement, of which there certainly has been plenty under Democratic Progressive Party (DPP) and KMT-led governments, the rationales for their proposed spending cuts lay bare the incoherent foreign policy of the KMT-led coalition. Introduced on the eve of US President Donald Trump’s inauguration, the KMT’s proposed budget is a terrible opening
The Chinese Nationalist Party (KMT) caucus in the Legislative Yuan has made an internal decision to freeze NT$1.8 billion (US$54.7 million) of the indigenous submarine project’s NT$2 billion budget. This means that up to 90 percent of the budget cannot be utilized. It would only be accessible if the legislature agrees to lift the freeze sometime in the future. However, for Taiwan to construct its own submarines, it must rely on foreign support for several key pieces of equipment and technology. These foreign supporters would also be forced to endure significant pressure, infiltration and influence from Beijing. In other words,
“I compare the Communist Party to my mother,” sings a student at a boarding school in a Tibetan region of China’s Qinghai province. “If faith has a color,” others at a different school sing, “it would surely be Chinese red.” In a major story for the New York Times this month, Chris Buckley wrote about the forced placement of hundreds of thousands of Tibetan children in boarding schools, where many suffer physical and psychological abuse. Separating these children from their families, the Chinese Communist Party (CCP) aims to substitute itself for their parents and for their religion. Buckley’s reporting is
Last week, the Chinese Nationalist Party (KMT) and the Taiwan People’s Party (TPP), together holding more than half of the legislative seats, cut about NT$94 billion (US$2.85 billion) from the yearly budget. The cuts include 60 percent of the government’s advertising budget, 10 percent of administrative expenses, 3 percent of the military budget, and 60 percent of the international travel, overseas education and training allowances. In addition, the two parties have proposed freezing the budgets of many ministries and departments, including NT$1.8 billion from the Ministry of National Defense’s Indigenous Defense Submarine program — 90 percent of the program’s proposed
RSS