The IBM “endorsement effect” has often shaped the computer industry over the years. In 1981, when IBM entered the personal computer business, the company decisively pushed an upstart technology into the mainstream.
In 2000, the open-source operating system Linux was viewed askance in many corporations as an oddball creation and even legally risky to use, since the open-source ethos prefers sharing ideas rather than owning them, but IBM endorsed Linux and poured money and people into accelerating the adoption of the open-source operating system.
APACHE SPARK
IBM was yesterday scheduled to announce a broadly similar move in big data software. The company is placing a large investment — contributing software developers, technology and education programs — behind an open-source project for real-time data analysis, called Apache Spark.
The commitment, according to Robert Picciano, senior vice president for IBM’s data analytics business, will amount to “hundreds of millions of dollars” a year.
In the big data software market, much of the attention and investment so far has been focused on Apache Hadoop and the companies distributing that open-source software, including Cloudera, Hortonworks and MapR.
Hadoop, put simply, is the software that makes it possible to handle and analyze vast volumes of all kinds of data. The technology came out of the pure Internet companies like Google and Yahoo, and is increasingly being used by mainstream companies, which want to do similar big data analysis in their businesses.
SPEED
However, if Hadoop opens the door to probing vast volumes of data, Spark promises speed. Real-time processing is essential for many applications, from analyzing sensor data streaming from machines to sales transactions on online marketplaces.
The Spark technology was developed at the Algorithms, Machines and People Lab at the University of California, Berkeley. A group from the Berkeley lab founded a company two years ago, Databricks, which offers Spark software as a cloud service.
Spark, Picciano said, is crucial technology that will make it possible to “really deliver on the promise of big data.”
That promise, he said, is to quickly gain insights from data to save time and costs, and to spot opportunities in fields like sales and new product development.
IBM said it will put more than 3,500 of its developers and researchers to work on Spark-related projects. It will contribute machine-learning technology to the open-source project, and embed Spark in IBM’s data analysis and commerce software. IBM will also offer Spark as a service on its programming platform for cloud software development, Bluemix. The company will open a Spark technology center in San Francisco to pursue Spark-based innovations.
PARTNERSHIPS
IBM also plans to partner with academic and private education organizations, including UC Berkeley’s AMPLab, DataCamp, Galvanize and Big Data University to teach Spark to as many as 1 million data engineers and data scientists.
By aligning itself with a popular open-source project, IBM, they said, hopes to attract more software engineers to use its big data software tools, too.
“It’s first and foremost a play for the minds — and hearts — of developers,” IDC analyst Dan Vesset said.
TECH BOOST: New TSMC wafer fabs in Arizona are to dramatically improve US advanced chip production, a report by market research firm TrendForce said With Taiwan Semiconductor Manufacturing Co (TSMC, 台積電) pouring large funds into Arizona, the US is expected to see an improvement in its status to become the second-largest maker of advanced semiconductors in 2027, Taipei-based market researcher TrendForce Corp (集邦科技) said in a report last week. TrendForce estimates the US would account for a 21 percent share in the global advanced integrated circuit (IC) production market by 2027, sharply up from the current 9 percent, as TSMC is investing US$65 billion to build three wafer fabs in Arizona, the report said. TrendForce defined the advanced chipmaking processes as the 7-nanometer process or more
China’s Huawei Technologies Co (華為) plans to start mass-producing its most advanced artificial intelligence (AI) chip in the first quarter of next year, even as it struggles to make enough chips due to US restrictions, two people familiar with the matter said. The telecoms conglomerate has sent samples of the Ascend 910C — its newest chip, meant to rival those made by US chipmaker Nvidia Corp — to some technology firms and started taking orders, the sources told Reuters. The 910C is being made by top Chinese contract chipmaker Semiconductor Manufacturing International Corp (SMIC, 中芯) on its N+2 process, but a lack
Who would not want a social media audience that grows without new content? During the three years she paused production of her short do-it-yourself (DIY) farmer’s lifestyle videos, Chinese vlogger Li Ziqi (李子柒), 34, has seen her YouTube subscribers increase to 20.2 million from about 14 million. While YouTube is banned in China, her fan base there — although not the size of YouTube’s MrBeast, who has 330 million subscribers — is close to 100 million across the country’s social media platforms Douyin (抖音), Sina Weibo (新浪微博) and Xiaohongshu (小紅書). When Li finally released new videos last week — ending what has
OPEN SCIENCE: International collaboration on math and science will persevere even if the incoming Trump administration imposes strict controls, Nvidia’s CEO said Nvidia Corp CEO Jensen Huang (黃仁勳) said on Saturday that global cooperation in technology would continue even if the incoming US administration imposes stricter export controls on advanced computing products. US president-elect Donald Trump, in his first term in office, imposed restrictions on the sale of US technology to China citing national security — a policy continued under US President Joe Biden. The curbs forced Nvidia, the world’s leading maker of chips used for artificial intelligence (AI) applications, to change its product lineup in China. The US chipmaking giant last week reported record-high quarterly revenue on the back of strong AI chip
RSS