Until recently, scanning the entire Internet, with its billions of unique addresses, was a slow and labor-intensive process. For example, in 2010 the Electronic Frontier Foundation conducted a scan to gather data on the use of encryption online. The process took two to three months.
A team of researchers at the University of Michigan believed they could do better. A lot better. On Friday, at the Usenix security conference in Washington, they announced ZMap, a tool that allows an ordinary server to scan every address on the Internet in just 44 minutes.
The EFF team used a tool called Nmap that sends a request to a machine and then listens for the recipient to reply. These requests can be conducted in parallel, but keeping records for each outstanding request still creates a lot of overhead, which slows down the scanning process.
In contrast, ZMap is “stateless,” meaning that it sends out requests and then forgets about them. Instead of keeping a list of oustanding requests, ZMap cleverly encodes identifying information in outgoing packets so that it will be able to identify responses. The lower overhead of this approach allows ZMap to send out packets more than 1,000 times faster than Nmap. So while an Internet-wide scan with Nmap takes weeks, ZMap can (with a gigabit network connection) scan the entire Internet in 44 minutes.
The ability to rapidly and cheaply scan the entire Internet opens up some fascinating new possibilities for Internet-wide research.
