<h1>1. Summary</h1><p>Ready incorporates three data sources in the current system to create the seven categories of performance indicators. The data includes FCC’s Form 477, <a href="https://registry.opendata.aws/speedtest-global-performance/" target="_blank" rel="noopener noreferrer">Ookla</a>’s global fixed broadband network performance, and <a href="https://www.measurementlab.net/data/" target="_blank" rel="noopener noreferrer">Measurement Lab</a> (M-Lab)’s Network Diagnostic Tool (NDT). This article explains the step-by-step methodology.&nbsp;</p><p><strong>You can use the data in Broadband.money’s maps to win IIJA BEAD grants.</strong></p><figure class="image"><img src="https://storage.googleapis.com/bbm-public-bucket/readyspeedrank2_a04c6e91be.png"></figure><p style="text-align:center;"><span class="text-small"><i>Fig. 1 A screen capture of current performance indicators shown in BBM. The highlighted census block is classified as “Arguably Unserved” by Ready. We provide detailed data from FCC's form 477, Measurement Lab's data, and Ookla to support this classification.</i></span></p><p>The following section first explains the method to aggregate speed test data to the Census Block Level, then explains the construction of the Ready Speed Rank.</p><h1>2. Method</h1><h2>2.1 Data Aggregation</h2><h3>2.1.1 Ookla</h3><p>The original Ookla global fixed broadband network performance data is distributed through vector tiles (end note). Each tile is approximately 2003.937 feet by 2003.937 feet at the equator. The data contains the average download speed, upload speed, and latency of all tests performed in the tile, number of tests, number of devices, and time when the data was delivered. Ready re-interpolates the data to Census Block (CB) level considering all tiles intersected with a census block. We currently provide a list of attributes for Ookla data for each of the four quarters of 2021. Noted that for the census blocks where no speed tests are available, Ready uses the speed test data interpolated from the larger Census Block Group (CBG) level instead. The resolution of the data can be found under the Speed Source attribute. In other words, we provide an indicator to differentiate if the data is a census block level estimate or a census block group estimate.</p><figure class="table"><table style="border:0.1px solid hsl(0, 0%, 60%);"><thead><tr><th style="background-color:hsl(0, 0%, 90%);border:0.1px solid hsl(0, 0%, 30%);">Attributes</th><th style="background-color:hsl(0, 0%, 90%);border:0.1px solid hsl(0, 0%, 30%);">Description</th></tr></thead><tbody><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Minimum Download Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Minimum download speed of all tests performed in the tiles that intersect with a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Maximum Download Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Maximum download speed of all tests performed in the tiles that intersect with a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Median Download Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Median download speed of all tests performed in the tiles that intersect with a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Minimum Upload Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Minimum upload speed of all tests performed in the tiles that intersect with a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Maximum Upload Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Maximum upload speed of all tests performed in the tiles that intersect with a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Median Upload Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Median upload speed of all tests performed in the tiles that intersect with a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Latency</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">The average latency of all tests performed within the tiles intersected with a census block, represented in milliseconds</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Speed Category Ookla</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Performance categories are summarized based on the median download and upload speed. A census block with a speed test lower than 25/3 is unserved, 100/20 underserved, and served otherwise.</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Number of Tests</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Total number of tests conducted within the tiles intersected with a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Number of Devices</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Total number of distinct devices participated the speed tests within the tiles intersected with a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Source</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Details on data extraction method, indicating if the data is directly extracted from the vector tiles or re-interpolated using the speed test data from the adjacent census block groups.</span></td></tr></tbody></table></figure><h3>2.1.2 M-Lab</h3><p>The M-Lab’s Network Diagnostic Tool (NDT) provides download speed, upload speed, latency, and packet loss rate metrics for each speed test result collected. NDT server collects TCP statistics using <a href="https://www.measurementlab.net/tests/tcp-info/" target="_blank" rel="noopener noreferrer">TCP_INFO</a>, via three protocols (in chronological order): <a href="https://www.measurementlab.net/tests/ndt/web100/" target="_blank" rel="noopener noreferrer">web100</a>, <a href="https://www.measurementlab.net/tests/ndt/ndt5/" target="_blank" rel="noopener noreferrer">ndt5</a>, and <a href="https://www.measurementlab.net/tests/ndt/ndt7/" target="_blank" rel="noopener noreferrer">ndt7</a>. Given both privacy concerns and the robustness of the data, M-Lab only provides the location and measured speeds of tests aggregated at certain geographic levels such as the county, census block group, or zip code. Ready first identifies census blocks with their counties and provides summary statistics of all tests taken within the county. Specifically, the number of speed tests taken within a census block is computed based on the proportion of the population of that census block to the population of its associated county. Similar to Ookla, Ready also provides quarterly updates to reflect the potential changes in Internet services.</p><p>When there are no speed tests captured within an area, Ready builds a prediction model to estimate the local speed test results using the socio-economic attributes from American Community Survey (5-year, 2015-2019) and historical speeds from both Ookla and M-Lab. The Speed Source attribute indicates if the data is collected directly from original speed test vendors or computed internally by Ready.</p><figure class="table"><table style="border:0.5px solid hsl(0, 0%, 0%);"><thead><tr><th style="background-color:hsl(0, 0%, 90%);border:0.1px solid hsl(0, 0%, 30%);">Attributes</th><th style="background-color:hsl(0, 0%, 90%);border:0.1px solid hsl(0, 0%, 30%);">Description</th></tr></thead><tbody><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Minimum Download Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Minimum download speed of all tests performed within a census block group or the smallest associated geographical unit</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Maximum Download Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Maximum download speed of all tests performed within a census block group or the smallest associated geographical unit</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Median Download Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Median download speed of all tests performed within a census block group or the smallest associated geographical unit</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Latency</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">The average latency of all tests performed within a census block group or the smallest associated geographical unit, represented in milliseconds.</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Packet Loss Rate</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">The average loss rate from the lifetime of the connection</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Minimum Upload Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Minimum upload speed of all tests performed within a census block group or the smallest associated geographical unit</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Median Upload Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Median download speed of all tests performed within a census block group or the smallest associated geographical unit</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Maximum Upload Speed</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Maximum download speed of all tests performed within a census block group or the smallest associated geographical unit</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Number of Tests</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Number of tests taken within a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Number of Devices</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Number of devices participating the speed test within a census block</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Source</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Details on data extraction method, indicating if the data is directly aggregated using the speed test data within each census block group or Ready predicts the performance with adjacent data available</span></td></tr><tr><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Speed Category M-Lab</span></td><td style="border:0.1px solid hsl(0, 0%, 30%);"><span class="text-small">Performance categories are summarized based on the median download and upload speed. A census block with a speed test lower than 25/3 is unserved, 100/20 underserved, and served otherwise.</span></td></tr></tbody></table></figure><h3>2.1.3 FCC’s Form 477</h3><p>FCC’s Form 477 (as of <strong>June 30, 2021</strong>) provides ISP’s advertised download speed and upload speed for residential and business services at the census block level. We only consider the residential service to create the Ready Speed Rank. The detailed specification of the data can be seen here: <a href="https://opendata.fcc.gov/Wireline/Fixed-Broadband-Deployment-Data-December-2020/hicn-aujz" target="_blank" rel="noopener noreferrer">Data as of June&nbsp;30, 2021</a>. (Noted that <a href="https://broadbandusa.maps.arcgis.com/apps/webappviewer/index.html?id=e2b4907376b548f892672ef6afbc0da5" target="_blank" rel="noopener noreferrer">NTIA's current broadband indicator map</a> uses the data as of Dec 31, 2020.) We do not include satellite providers in this consideration. In addition, to make sure that the maximum download and upload speed is provided by the same provider, we first sort the data by providers' advertised download speed to find the highest download speed and then report the associated upload speed accordingly.&nbsp;</p><h2><strong>2.2 Construct the Ready Speed Rank</strong></h2><p>We present the logic behind the Ready Speed Rank in Fig 2. According to the <a href="https://broadbandusa.ntia.doc.gov/sites/default/files/2022-05/BEAD%20NOFO.pdf" target="_blank" rel="noopener noreferrer">NOFO</a>, unserved locations lack access to Reliable Broadband Service at speeds of at least 25 Mbps downstream and 3 Mbps upstream and low latency levels to support real-time, interactive applications. Underserved locations are those without broadband service offering speeds of 100 Mbps downstream/20 Mbps upstream. We adopt a decision tree method to create the Ready Speed Rank categories.</p><figure class="image"><img src="https://storage.googleapis.com/bbm-public-bucket/Ready_Speed_Rank_e7e0121ac5.png"></figure><p style="text-align:center;"><span class="text-small"><i>Fig. 2 Ready Speed Rank Decision Tree Process. “FCC Ad Speed” stands for advertised consumer download and upload speed from FCC's form 477.&nbsp;</i></span></p><h2>2.3 Differences between Ookla and M-Lab’s measure</h2><p>According to M-Lab official documentation, M-Lab’s “NDT is a <i>single stream</i> performance measurement of a connection’s capacity for “bulk transport”, which measures the “inter” part of the internet connection.</p><p>On the contrary, Ookla’s <a href="http://speedtest.net/">speedtest.net</a> provides data from servers near the user. In short, M-Lab represents the performance of the whole network, while Ookla provides better evidence on the last-mile connection. Ready provides both measures as a comparison to show the potential places needing better Internet services.</p><h2>2.4 Data Verification</h2><p>Estimating Internet performance is a task currently with no ground truth. To demonstrate the validity of our speed rank categories, we produce a case study that compares address-level Internet performance survey results and our performance indicators. For detailed analysis, please refer to this post <a href="https://broadband.money/blog/why-ready-speed-rank" target="_blank" rel="noopener noreferrer">here</a>.&nbsp;</p><h1>3. Known Limitations</h1><h2>3.1 Location precision</h2><p>Although M-Lab provides geolocation of each speed test’s associated IP address, the location (pre-aggregated at zip code level) might not reflect the actual location of where the speed test was conducted.</p><h2>3.2 Differentiation between business vs. residential</h2><p>The speed test data from both Ookla and M-Lab do not indicate if the data was collected from a commercial building or a residential building. This may lead to overestimates of performance for residential areas with office buildings nearby.</p><h2>3.3 Number of tests conducted</h2><p>Many census blocks do not have enough tests conducted (we consider at least 30 samples as “enough"). We suggest accounting for this in any decision-making, especially in low-income communities.</p><h2>3.4 No data or no services</h2><p>Speed tests result cannot tell us whether a location is <i>short of tests</i> or <i>has no Internet services</i>. Ready suggests that local communities should coordinate with state-level broadband offices and dedicate additional resources to locations with no tests reported.</p><h2>3.5 Unreported Provider in FCC’s Form 477</h2><p>The current advertised speed from FCC is limited to the latest edition of the released Form 477 data. If a provider hasn’t submitted its latest Form 477, we have not yet incorporated their services. Ready will further enhance this aspect once the data is validated.</p><p>&nbsp;</p><h1>End Notes</h1><ol><li>Vector tile is a way to deliver geographic data in small chunks to a browser or other client application.</li><li>The single stream “speed” metric doesn’t measure link capacity but rather how well a single TCP stream can take advantage of that capacity.</li></ol>

Methodology of creating the Ready Speed Rank using multiple data sources.

Broadband Grant Expert

Zhuangyuan Fan

Lessons

Articles

Enroll for free today