5/9/2023 0 Comments Spacenet challenge![]() Every top-5 competitor except for number13 trained every model on imagery from all of the collects combined, and then averaged the results from all of the models. ![]() Algorithm ensemblesĮvery competitor in the top 5 used an “ensemble” of CNNs, i.e., they trained multiple models and combined the results for their final output. Number13 used Mask-RCNN, a combined object detection/classification/segmentation algorithm. The vast majority of competitors’ algorithms performed semantic segmentation using ensembles of U-Nets with a variety of state-of-the-art image classification encoders. Let’s go through a couple of details of the competitors’ solutions: Neural net architectures Inference was run using only one of the four GPUs. *: Training time and inference time were measured on a server with 16 Intel Xeon 3.5 GHz CPUs, 4 Titan Xp GPUs with 12 GB GPU memory each, 256 GB of RAM, and a 2 TB SSD for data storage. Let’s dive into the details! Winners’ algorithm performanceĪ summary of the winners’ results alongside the baseline model are in the table below.Ī summary of winners’ algorithms. 68 seconds per square km of prediction, this may represent the difference between a deployable solution and an algorithm that is only academically relevant. ![]() This came at a substantial computing cost: the winning algorithm takes more than 10 times longer to identify buildings in new images than the 5th place competitor’s algorithm! With a difference of 6.5 vs. The winning algorithm, an ensemble of 28 convolutional neural networks (CNNs) with a gradient boosting machine (GBM) filter to remove bad buildings yielded only a 5% improvement over the #5 algorithm, which comprised only 3 CNNs. Big ensembles of models help, but the juice may not always be worth the squeeze If you want to use algorithms to identify objects in very off-nadir imagery, be aware of these performance limitations. In these perfectly cloudless, high-resolution Worldview 2 images over Atlanta, even when we restrict to the images looking nearly straight down (look angle of <25 degrees off-nadir, the “nadir” set), the best algorithm still missed about 21% of buildings.įor every six predictions the winner made from very off-nadir images, about four corresponded to buildings. Key takeawaysīefore we get into the nitty-gritty, here are our main takeaways from running and scoring the challenge: Building detection in “ideal” imagery is still far from perfect We had 238 registrants and 21 competitors beating our baseline model in this extremely challenging competition. Introducing the SpaceNet Off-Nadir Imagery DatasetĪ Baseline Model for the SpaceNet Off-Nadir Building Detection ChallengeĬhallenges with the SpaceNet 4 off-nadir satellite imagery For more details about the challenge (or if you’re unfamiliar with that terminology), check out these earlier posts describing the competition: The data comprised both South-facing and North-facing collects with look angle ranging from 7 degrees off-nadir to a whopping 54 degrees off-nadir. In Round 4 of the SpaceNet Challenge we asked competitors to identify building footprints from 27 different satellite collects taken during a single pass of a WorldView 2 satellite over Atlanta. This is the first post in a series where we’ll dig into the competitors’ solutions to find out how they applied machine learning to identify buildings in overhead imagery taken at look angles from 7 to 54 degrees off-nadir. Their solutions represented a 1.5-fold improvement over our initial baseline model’s performance. Check out this GitHub repository for their solution algorithm code. The SpaceNet Challenge: Round 4, Off-Nadir Building Footprint Extraction hosted by TopCoder is complete! Congratulations to cannab, selim_sef, MaksimovKA, number13, and XD_XD for taking home the top 5 prizes. SpaceNet’s mission is to accelerate geospatial machine learning and is supported by the SpaceNet LLC member organizations.
0 Comments
Leave a Reply. |