Mapping the Digital Divide

Physicist Les Cottrell is the meteorologist of Internet weather. His project tests the strength of Internet connections around the world—and finds Africa lagging farther and farther behind.
By Madolyn Bowman Rogers

Photo-illustration: Sandbox Studio

Sandwiched between YouTube videos of laughing babies and exploding fountains of Mentos and Diet Coke, you can find a 45-second animation of Africa’s “Internet weather.” The continent is bright yellow, and as the video runs, pale red dots flicker across its surface. A voiceover with a sprightly British accent informs the viewer that the dots represent Internet connectivity measured from April 2007 to March 2008. The paler the dot, the slower the connection, and the more the dots flicker, the more unstable the connection is. While the dots flutter dimly, the voice informs us that “Africa’s performance is the worst in the world, and is 10 or more years behind Europe and the US.”

The Internet has ushered in an age of global connectivity. People chat with friends across the world and get news instantly from the other side of the planet. Business partnerships and trade flourish across continents. Science is especially dependent on international participation. In the world of high-energy physics, cutting-edge projects like the Large Hadron Collider are possible only because of the efforts of scientists around the globe.

With so much depending on international connections, it seems someone should be monitoring just how reliable those connections are. For the last 13-plus years, that job has fallen to the PingER project, based at the Stanford Linear Accelerator Center in California. PingER, short for Ping End-to-end Reporting, measures the speed and reliability of Internet connections between institutions around the world. The project began as a resource for high-energy physics, but all the measurements are publicly available, and in recent years numerous universities and scientific organizations have discovered the value of the data.

PingER data show that not everyone can participate equally in the global culture. Many countries are left behind, their residents isolated behind slow and unreliable connections. The PingER analysis has inspired the leaders of several countries, including India, Brazil, Romania, and Pakistan, to invest in major upgrades to communications equipment. Other places, such as most of Africa, have yet to catch up. Meanwhile, demands on the global network are increasing exponentially, and the need for someone to monitor performance and detect problems is more crucial than ever.

The project, however, maintains a tenuous existence. It’s a largely unfunded program run by a handful of computer scientists and physicists, and it relies heavily on volunteer labor. Its existence depends almost entirely on support from the Pakistan National University of Sciences and Technology and the dedication of one man, Les Cottrell.

The Internet’s weatherman

Cottrell, who describes himself as a “renegade physicist,” has been managing the computer network at SLAC for roughly 40 years. He is the narrator of the Africa video, the meteorologist of Internet weather. He started the PingER project in 1994 as a way to improve communication between high-energy physics sites around the world, which were becoming increasingly dependent on the Internet.

“Occasionally something would fail,” Cottrell says. “People would immediately run over to the phone network and say ‘What’s wrong? What have you done?’ And we didn’t have much of an idea.”

Cottrell is tall and lanky, glasses perched on his nose beneath a mop of curly white hair, and speaks with an elegant British accent. He is refreshingly direct; one of his reports states that a data packet loss of 2.5 percent “will result in Voice Over Internet Protocols becoming slightly annoying every 30 seconds or so.” Faced with physicists clamoring for answers whenever there was a failure, Cottrell decided he needed to make measurements. “The old mantra is you can’t manage what you can’t measure,” he says.

Cottrell made use of a common Internet data packet, the ping, a tool that comes packaged with every computer. It works this way: One computer, acting as a “monitoring host,” sends a ping to a remote computer, which then sends it back. The round-trip time reveals the connection speed between the computers, including the time it takes the remote host to process the ping. The PingER project sends 10 pings every 30 minutes to each remote site. Long round-trip times indicate congestion or poor routing. In very congested traffic, packets may be lost in transit. Remote hosts may also show variable performance, doing well in processing some packets and poorly with others. If no packets at all come back, the remote host may be down.

“So just with this trivial measurement, you’ve measured round-trip time, loss, jitter, and reachability,” Cottrell says. The data is analyzed and stored at SLAC and Fermilab, and the results are made publicly available. When the project started, a single monitoring host at SLAC was measuring about 40 sites. The PingER project now has 44 monitoring hosts in 22 countries that send pings to more than 700 sites in 159 countries.

Using data to drive progress

One of the first organizations to use the data was the International Committee on Future Accelerators (ICFA), a group formed in 1976 to promote worldwide cooperation in accelerator building and research. In 1996 the group issued a statement recognizing the importance of communications to the field of high-energy physic and urging countries to upgrade their equipment to ensure adequate bandwidth.

“Occasionally something would fail,” Cottrell says. “People would immediately run over to the phone network and say ‘What’s wrong? What have you done?’ And we didn’t have much of an idea.” Photo: Brad Plummer, SLAC

“Our whole field is based on global collaboration,” says Harvey Newman, a physicist at the California Institute of Technology and chair of the group’s Standing Committee on Interregional Connectivity. “The Large Hadron Collider in particular has penetrated to every region of the world. Without connectivity, you can’t really be a partner in these experiments.”

Newman’s committee looks at PingER data every year, as well as complementary data on the status and growth of the Internet, to discover countries where infrastructure isn’t keeping up with demand. The committee then shares this data with major research and education networks in hopes of encouraging improvements. Where they’ve done so, “it’s really helped to spark progress in that country,” Newman says. A February 2004 ICFA meeting in Rio de Janeiro triggered improvements that have taken Brazil from a relatively primitive state of connectivity to the leading country in Latin America in just four years.

In 1999, Newman’s committee visited the National University of Sciences and Technology in Pakistan and showed it the PingER data, beginning a slow process of improvement. In 2001, Cottrell went to Pakistan to present his data personally.

“I even met Musharraf. Mind you, I was too much in awe to say much,” Cottrell says. “I shook his hand and said, ‘Very nice of you to have us here.’”

Finding bottlenecks

Cottrell also met with Atta ur Rahman, the head of higher education in Pakistan, and showed him data that demonstrated that even though the backbone network in Pakistan was quite good at the time, the universities had terrible connections—only 1 megabit per second connection speed for the entire institution, or a factor of 1000 to 10,000 times less than the connections typically available to universities in the United States. The PingER data made it clear that the problem lay in the last mile of connections to the universities, which were supplied by copper wire. Rahman jumped on this, Cottrell says, and Pakistan has since upgraded the connections to fiber.

Pakistan and SLAC began a fruitful collaboration, formalized in a 2004 memorandum of understanding that provided crucial funding for the PingER project. Pakistani students in the School of Electrical Engineering and Computer Sciences help develop software for PingER. Those who do well are invited to come to SLAC for one year to help analyze PingER data.

“It’s a wonderful opportunity for students in Pakistan. They’re usually not exposed to such real-world problems,” says Umar Kalim, a computer scientist at the School of Electrical Engineering who came to SLAC in February 2008 to work with Cottrell on Internet performance monitoring.

The PingER project “has created a snapshot of how the network has evolved in Pakistan.” Kalim says. “Compared to its neighbors, Pakistan has made tremendous progress.”

Africa: left in the dark

The PingER project has also drawn the interest of the Abdus Salam International Centre for Theoretical Physics in Trieste, Italy, whose mission is to support science in developing countries.

“One of the main problems for scientists in developing countries is lack of bandwidth,” says Enrique Canessa, a physicist at the center. “To be connected today is essential.”

The center invited Cottrell to present his data at a 2002 roundtable with scientists, politicians, and educators from across the globe. As a result, numerous universities in Latin America, Asia, and Africa agreed to participate in the PingER project, extending its reach to much of the developing world. ICTP began to track connectivity between Italy and 40 countries in Africa whose populations make up more than 80 percent of the continent’s inhabitants.

The data revealed a continent in the dark. Many universities in Africa have less bandwidth than a well-connected home in Europe does. Only 3.6% of Africa’s one billion people have access to the Internet, and the cost of broadband Internet service can be as high as $8000 per month. In many parts of Africa, for example in Uganda, all Internet connections go through satellite links. Because the signal has to make a 48,000 mile round-trip to the satellite and back, it takes four times as long as it would to travel through a landline connection.

“They always talked about Africa as being the ‘Dark Continent.’ Actually it’s incredibly true,” Cottrell says. “The really concerning thing is not only are they a long way behind, probably around 12-15 years behind, but they’re not catching up. It’s getting much worse.”

If current trends continue, by 2016 connection performance between North America and Africa will be 1000 times worse than between North America and Europe, according to Cottrell’s most recent report.

Africa’s near-absence from the global Internet community is a loss to the world, Cottrell says. He believes it’s important “to get these excellent minds leverage so they can do something useful.”

Scientific organizations such as the International Heliophysical Year depend on having good connectivity in Africa. The group studies sunspots, which can throw off GPS tracking, and needs to make its measurements from the equator. Africa has the most equatorial land mass of any continent, but without good connections, scientists can’t upload the data they generate there.

In some parts of Africa, the connections are better, and there is enough infrastructure to build on. “The data tell us which countries have a chance of moving forward,” Cottrell says.

Outlook: cloudy

The hard data provided by PingER is crucial for convincing politicians and decision-makers to invest money in improving connectivity, say Canessa and co-worker Marco Zennaro from ICTP. In April 2008, ICTP and Cottrell collaborated on the video of Internet weather in Africa and posted it on YouTube.

“Now there is more global awareness of the lack of connectivity and its importance,” Canessa says.

Still, the project remains largely unfunded. PingER is not part of Cottrell’s job description at SLAC, and he donates much of the time he spends on it. The funding from Pakistan pays for software development, Cottrell says, but the PingER project also requires a lot of day-to-day maintenance. Remote hosts regularly go offline, for example if they move or install a firewall, and Cottrell or one of his helpers must contact the host to re-establish the connection.

“The project is designed to need very barebones effort,” Cottrell says. “We’ll try to keep it running at a very low level.” But without more funding, he says, he may no longer be able to produce reports summarizing PingER findings. And if all the funding gets cut, or when he himself retires, “I suppose it would go away entirely. I’d be disappointed. I think it’s very valuable, in particular for Africa.”

So PingER’s future remains cloudy; but for now, the Internet’s meteorologist will keep tracking the weather.

The rendering above displays the relative densities of Internet connectivity across the globe. The stronger the contrast, the more connectivity there is. It is immediately obvious that North America and Europe are considerably more connected than Africa or South America. However, it is important to note that this map reflects only density of connections, not usage; hundreds of people may share a single connection in an Internet cafe, often the only access people have in developing nations. Image courtesy of Chris Harrison, Carnegie Mellon University

Click here to download the pdf version of this article.

Send a letter to the editor