1. Summary

Ready incorporates three data sources in the current system to create the seven categories of performance indicators. The data includes FCC’s Form 477, Ookla’s global fixed broadband network performance, and Measurement Lab (M-Lab)’s Network Diagnostic Tool (NDT). This article explains the step-by-step methodology.

You can use the data in Broadband.money’s maps to win IIJA BEAD grants.

Fig. 1 A screen capture of current performance indicators shown in BBM. The highlighted census block is classified as “Arguably Unserved” by Ready. We provide detailed data from FCC's form 477, Measurement Lab's data, and Ookla to support this classification.

The following section first explains the method to aggregate speed test data to the Census Block Level, then explains the construction of the Ready Speed Rank.

2. Method

2.1 Data Aggregation

2.1.1 Ookla

The original Ookla global fixed broadband network performance data is distributed through vector tiles (end note). Each tile is approximately 2003.937 feet by 2003.937 feet at the equator. The data contains the average download speed, upload speed, and latency of all tests performed in the tile, number of tests, number of devices, and time when the data was delivered. Ready re-interpolates the data to Census Block (CB) level considering all tiles intersected with a census block. We currently provide a list of attributes for Ookla data for each of the four quarters of 2021. Noted that for the census blocks where no speed tests are available, Ready uses the speed test data interpolated from the larger Census Block Group (CBG) level instead. The resolution of the data can be found under the Speed Source attribute. In other words, we provide an indicator to differentiate if the data is a census block level estimate or a census block group estimate.

Attributes	Description
Minimum Download Speed	Minimum download speed of all tests performed in the tiles that intersect with a census block
Maximum Download Speed	Maximum download speed of all tests performed in the tiles that intersect with a census block
Median Download Speed	Median download speed of all tests performed in the tiles that intersect with a census block
Minimum Upload Speed	Minimum upload speed of all tests performed in the tiles that intersect with a census block
Maximum Upload Speed	Maximum upload speed of all tests performed in the tiles that intersect with a census block
Median Upload Speed	Median upload speed of all tests performed in the tiles that intersect with a census block
Latency	The average latency of all tests performed within the tiles intersected with a census block, represented in milliseconds
Speed Category Ookla	Performance categories are summarized based on the median download and upload speed. A census block with a speed test lower than 25/3 is unserved, 100/20 underserved, and served otherwise.
Number of Tests	Total number of tests conducted within the tiles intersected with a census block
Number of Devices	Total number of distinct devices participated the speed tests within the tiles intersected with a census block
Source	Details on data extraction method, indicating if the data is directly extracted from the vector tiles or re-interpolated using the speed test data from the adjacent census block groups.

2.1.2 M-Lab

The M-Lab’s Network Diagnostic Tool (NDT) provides download speed, upload speed, latency, and packet loss rate metrics for each speed test result collected. NDT server collects TCP statistics using TCP_INFO, via three protocols (in chronological order): web100, ndt5, and ndt7. Given both privacy concerns and the robustness of the data, M-Lab only provides the location and measured speeds of tests aggregated at certain geographic levels such as the county, census block group, or zip code. Ready first identifies census blocks with their counties and provides summary statistics of all tests taken within the county. Specifically, the number of speed tests taken within a census block is computed based on the proportion of the population of that census block to the population of its associated county. Similar to Ookla, Ready also provides quarterly updates to reflect the potential changes in Internet services.

When there are no speed tests captured within an area, Ready builds a prediction model to estimate the local speed test results using the socio-economic attributes from American Community Survey (5-year, 2015-2019) and historical speeds from both Ookla and M-Lab. The Speed Source attribute indicates if the data is collected directly from original speed test vendors or computed internally by Ready.

Attributes	Description
Minimum Download Speed	Minimum download speed of all tests performed within a census block group or the smallest associated geographical unit
Maximum Download Speed	Maximum download speed of all tests performed within a census block group or the smallest associated geographical unit
Median Download Speed	Median download speed of all tests performed within a census block group or the smallest associated geographical unit
Latency	The average latency of all tests performed within a census block group or the smallest associated geographical unit, represented in milliseconds.
Packet Loss Rate	The average loss rate from the lifetime of the connection
Minimum Upload Speed	Minimum upload speed of all tests performed within a census block group or the smallest associated geographical unit
Median Upload Speed	Median download speed of all tests performed within a census block group or the smallest associated geographical unit
Maximum Upload Speed	Maximum download speed of all tests performed within a census block group or the smallest associated geographical unit
Number of Tests	Number of tests taken within a census block
Number of Devices	Number of devices participating the speed test within a census block
Source	Details on data extraction method, indicating if the data is directly aggregated using the speed test data within each census block group or Ready predicts the performance with adjacent data available
Speed Category M-Lab	Performance categories are summarized based on the median download and upload speed. A census block with a speed test lower than 25/3 is unserved, 100/20 underserved, and served otherwise.

2.1.3 FCC’s Form 477

FCC’s Form 477 (as of June 30, 2021) provides ISP’s advertised download speed and upload speed for residential and business services at the census block level. We only consider the residential service to create the Ready Speed Rank. The detailed specification of the data can be seen here: Data as of June 30, 2021. (Noted that NTIA's current broadband indicator map uses the data as of Dec 31, 2020.) We do not include satellite providers in this consideration. In addition, to make sure that the maximum download and upload speed is provided by the same provider, we first sort the data by providers' advertised download speed to find the highest download speed and then report the associated upload speed accordingly.

2.2 Construct the Ready Speed Rank

We present the logic behind the Ready Speed Rank in Fig 2. According to the NOFO, unserved locations lack access to Reliable Broadband Service at speeds of at least 25 Mbps downstream and 3 Mbps upstream and low latency levels to support real-time, interactive applications. Underserved locations are those without broadband service offering speeds of 100 Mbps downstream/20 Mbps upstream. We adopt a decision tree method to create the Ready Speed Rank categories.

Fig. 2 Ready Speed Rank Decision Tree Process. “FCC Ad Speed” stands for advertised consumer download and upload speed from FCC's form 477.

2.3 Differences between Ookla and M-Lab’s measure

According to M-Lab official documentation, M-Lab’s “NDT is a single stream performance measurement of a connection’s capacity for “bulk transport”, which measures the “inter” part of the internet connection.

On the contrary, Ookla’s speedtest.net provides data from servers near the user. In short, M-Lab represents the performance of the whole network, while Ookla provides better evidence on the last-mile connection. Ready provides both measures as a comparison to show the potential places needing better Internet services.

2.4 Data Verification

Estimating Internet performance is a task currently with no ground truth. To demonstrate the validity of our speed rank categories, we produce a case study that compares address-level Internet performance survey results and our performance indicators. For detailed analysis, please refer to this post here.

3. Known Limitations

3.1 Location precision

Although M-Lab provides geolocation of each speed test’s associated IP address, the location (pre-aggregated at zip code level) might not reflect the actual location of where the speed test was conducted.

3.2 Differentiation between business vs. residential

The speed test data from both Ookla and M-Lab do not indicate if the data was collected from a commercial building or a residential building. This may lead to overestimates of performance for residential areas with office buildings nearby.

3.3 Number of tests conducted

Many census blocks do not have enough tests conducted (we consider at least 30 samples as “enough"). We suggest accounting for this in any decision-making, especially in low-income communities.

3.4 No data or no services

Speed tests result cannot tell us whether a location is short of tests or has no Internet services. Ready suggests that local communities should coordinate with state-level broadband offices and dedicate additional resources to locations with no tests reported.

3.5 Unreported Provider in FCC’s Form 477

The current advertised speed from FCC is limited to the latest edition of the released Form 477 data. If a provider hasn’t submitted its latest Form 477, we have not yet incorporated their services. Ready will further enhance this aspect once the data is validated.

End Notes

Vector tile is a way to deliver geographic data in small chunks to a browser or other client application.
The single stream “speed” metric doesn’t measure link capacity but rather how well a single TCP stream can take advantage of that capacity.

BEAD Lessons

How Ready Speed Rank Works

Enroll for free today