Introduction

We broke down the process of developing a geospatial AI solution for the detection of roof faces into the following phases:

• Selection and import of a suitable roof face dataset
• Creation of an appropriate segmentation config file using geolibs
• Leveraging geotrain: a component of geolibs for parallel training of multiple models
• Tracking of experiments’ progress and comparison between multiple experiments through GeoEngine platform

Dataset Selection

We selected the Roof Information Dataset(RID)[1]: a standard remote-sensing dataset of roof faces and superstructures containing 1880 annotated roofs. The images cover an area of 1.5 km2, excluding overlap, and 4.9 km2 in total. The roof segment annotations include 4520 polygons, and their classes were derived by its azimuths. Lee et al. selected 16 azimuth classes, for example, south, south–south–west, and south–west, plus a class for flat roofs and one for trees. Usually, roof orientations are biased towards north, south, east, and west for architectural reasons. Therefore, we provided code to classify roof segments in three different ways.

As the first stage of solution generation, we imported this dataset into GeoEngine platform here for visualization and exploration purposes, and generated an export for model training in later stages.

Configuration File Creation

We leverage geolibs: our proprietary geospatial AI library that acts as a common interface between multiple open-mm-lab libraries (e.g. mmpretrain, mmsegmentation, mmdetection, etc), specialized towards geospatial AI solutions’ development. geolibs provides an interactive interface for configuration file generation, wherein we only need to specify our registered organization, GeoEngine project, export version, and public URL of the base mm configuration file from GitHub. Below is an example of how we interface with geolibs for config generation:

```

(geolibs) akash@berlin:~$ engine projects
🌐 Which type of projects do you want to see? From my organization

🏢 Please select an organization: Granular

📁 Please select a project: Roof Face Segmentation

📂 Do you want to see Mineral Stock details? No

🚧 Do you want to see Mineral Stock exports? Yes

🚧 Please select an export Original Export has coco format and is a designated export of Mineral Stock: Fold 1

Do you want to generate a geolibs experiment configuration for Granular's Mineral Stock with Original Export export?
🤓  Yes

❓ Please select a question Mineral Present

🔗 This is a segmentation task. Please go to https://github.com/open-mmlab/mmsegmentation/tree/main/configs and fin
d a suitable config file and paste its URL here:  https://github.com/open-mmlab/mmsegmentation/blob/main/configs/segformer/segformer_mit-b0_8xb1-160k_cityscapes-1024x1024.py

📩 Please enter location to save the experiment configuration file ./
Saved to ./segformer_mit-b0_8xb1-160k-1024x1024.py

You can use the config file using geolibs:
python tools/train.py ./segformer_mit-b0_8xb1-160k-1024x1024.py


You can download the dataset using following command. Please make sure it is inside data folder.
mkdir Roof_Face_Segmentation 
mkdir Roof_Face_Segmentation/rasters Roof_Face_Segmentation/vectors 
gsutil -m cp -n -r gs://geoengine-dataset-rid/rasters/raw Roof_Face_Segmentation/rasters/ 
gsutil -m cp -n -r gs://geoengine-dataset-rid/vectors/Fold_1-20230808T044502 Roof_Face_Segmentation/vectors/

As is visible, geolibs creates a config file based on our URL: https://github.com/open-mmlab/mmsegmentation/blob/main/configs/segformer/segformer_mit-b0_8xb1-160k_cityscapes-1024x1024.py in the current directory, provides instructions to download the export version of our choice, and instructions to train our model. We make minute changes to our config file to be able to properly ingest our dataset and set the environment for training.

Model Training

geolibs provides a simple command: geotrain to take in the config file path as an argument and initiate training. For our specific use case, we invoked the following command:

(geolibs) akash@berlin:~$ geotrain segformer_mit-b0_8xb1-160k-1024x1024.py

This initiates training in our environment. For the roof face solution, we instantiated multiple EC2 instances, set up geolibs, used its interactive interface to generate configuration files for various model types, and ran multiple experiments in parallel.

Experiment Tracking

geolibs is integrated with engine CLI, a library that interfaces with the GeoEngine platform to track the progress of experiments. We configured engine configs in our individual config files to track and visualize experiment statistics in the platform here. GeoEngine also allows us to select multiple experiments for comparison.

Roof Face Experiments in GeoEngine Platform

Results

We tabulate the results of our experiments below:

Based on the table above, the experiment wise-gannet-5 has the best mIoU and mAcc values of 55.57 and 68.42 respectively, so we select this experiment to be deployed as a roof face solution.

References

[1]: Krapf, S., Bogenrieder, L., Netzler, F., Balke, G., and Lienkamp, M. 2022. RID: Roof Information Dataset for Computer Vision-Based Photovoltaic Potential Assessment. *Remote Sensing, 14*.