Using computer vision for the automatic classification of building facades

In the realm of sustainable architecture, the integration of technology and design has become a transformative force. Building facades significantly impact a structure's energy efficiency, environmental footprint, and aesthetic appeal. This thesis explores the intersection of artificial intelligence (AI) and sustainable design principles, presenting a novel approach to recognize building facades' primary material through semantic segmentation's precision. Selecting appropriate facade materials is crucial in the face of escalating climate concerns and the imperative to curtail carbon emissions. Architects, engineers, and urban planners must embrace data-driven solutions that mitigate the environmental impacts of built environments. During an examination of assessment databases across multiple US cities, it was noted that a significant amount of data is available on building addresses throughout the urban landscape. However, there is a noticeable lack of information about the exterior construction materials used in these buildings. This highlighted disparity has brought about the need for an advanced AI architecture capable of effectively mapping out the intricate distribution of materials within a city. Semantic segmentation, a subfield of computer vision, offers an unprecedented avenue to revolutionize the evaluation, selection, and incorporation of facade materials in construction projects. It empowers machines to distinguish between elements within an image at a pixel-level granularity. By harnessing the power of deep learning algorithms, an AI system can discern and categorize individual components of building facades, facilitating the identification of the predominant material with exceptional accuracy. The aim is to effectively map the material composition of urban landscapes by identifying the primary material of a building's facade, making it possible to extract precise distribution patterns of building materials within a city and recognize the main elements of the facades, such as windows, doors, and roofs. The proposed architecture leverages the NVIDIA HIERARCHICAL MULTI-SCALE ATTENTION architecture to extract features from input images and generate accurate predictions. The resulting model is trained and evaluated on a dataset of annotated building facade images collected from various sources such as Google Street View and manual annotations. The trained network can distinguish eight materials: brick, stucco, vinyl, concrete, glass, wood, asbestos, and asphalt. These are the primary materials on which buildings in the United States are made and the network reaches an accuracy of 92% over different test sets based on different cities, showing interesting results.