Automated Real-World Sustainability Data Generation from Images of Buildings

Read original: arXiv:2405.18064 - Published 8/29/2024 by Peter J Bentley, Soo Ling Lim, Rajat Mathur, Sid Narang

📊

Overview

When building data is unavailable, it becomes difficult to determine how to improve a building's carbon emissions.
The researchers show that a Large Language Model with appropriate prompting and domain knowledge can estimate building features relevant for sustainability calculations, using only a set of images.
They compare their novel image-to-data method with real building data for 47 apartments and achieve accuracy better than a human performing the same task.
The method can also generate tailored recommendations to the owner on how best to improve their properties, and the researchers discuss ways to scale the approach.

Plain English Explanation

Improving the sustainability of buildings is an important goal, but it often requires detailed data about the building's features, such as its size, materials, and energy usage. However, this data is not always available, especially for older or smaller buildings.

To address this challenge, the researchers in this study used a Large Language Model, a type of artificial intelligence system trained on vast amounts of text data. By carefully crafting the prompts and providing the model with relevant domain knowledge, they were able to estimate a range of building features relevant for sustainability calculations, using only a set of images of the building.

The researchers compared the estimates from their image-to-data method with the actual data for 47 apartment buildings, and found that their approach was more accurate than a human performing the same task. This suggests that their technique could be a valuable tool for assessing the sustainability of buildings, even when detailed data is lacking.

Moreover, the researchers found that their method can also generate tailored recommendations for the building owner on how to best improve the property's sustainability. This could be a powerful way to help building owners take concrete steps to reduce their carbon footprint.

The researchers also discuss ways to scale up their approach, so that it could be applied to a large number of buildings and help drive sustainability efforts on a broader scale. This could be particularly valuable for Automated National Urban Map Extraction, Scalability of Building Component Data Annotation, and General Remote Sensing Image Building Detection tasks, as well as Automated Building Archetype Generation and Large-Scale Urban Building Function Identification.

Technical Explanation

The researchers in this study developed a novel method to estimate a range of building features relevant for sustainability calculations, using only a set of images of the building. They employed a Large Language Model, which is a type of artificial intelligence system that has been trained on vast amounts of text data, and used appropriate prompting and domain knowledge to guide the model's predictions.

Furthermore, the researchers found that their method can also generate tailored recommendations for the building owner on how to best improve the property's sustainability. This could be a powerful way to help building owners take concrete steps to reduce their carbon footprint.

The researchers also discuss ways to scale up their approach, so that it could be applied to a large number of buildings and help drive sustainability efforts on a broader scale. This could involve techniques like Automated National Urban Map Extraction, Scalability of Building Component Data Annotation, and General Remote Sensing Image Building Detection, as well as Automated Building Archetype Generation and Large-Scale Urban Building Function Identification.

Critical Analysis

The researchers acknowledge several caveats and limitations of their approach. For example, they note that their method has only been tested on a relatively small sample of 47 apartment buildings, and that further validation on a larger and more diverse dataset would be necessary to fully assess its capabilities.

Additionally, the researchers point out that their method relies on the availability of high-quality building images, which may not always be readily available, particularly for older or more obscure properties. They suggest that incorporating additional data sources, such as satellite imagery or building records, could help to overcome this limitation.

Another potential concern is the extent to which the model's performance may be influenced by biases or inaccuracies in the training data used to develop the Large Language Model. The researchers acknowledge that this is an area that requires further investigation and mitigation strategies.

Despite these caveats, the researchers' work represents a promising step forward in the use of artificial intelligence to support building sustainability efforts, even in the absence of detailed data. As the researchers suggest, further refinement and scaling of their approach could lead to significant advancements in the field.

Conclusion

This study demonstrates the potential of using a Large Language Model with appropriate prompting and domain knowledge to estimate building features relevant for sustainability calculations, even when detailed data is unavailable. The researchers' novel image-to-data method achieved accuracy better than a human performing the same task, and they also showed that the approach can generate tailored recommendations to help building owners improve their properties.

While the researchers acknowledge several limitations and areas for further research, their work represents an important step forward in the use of artificial intelligence to support building sustainability efforts. By scaling up their approach, the researchers hope to drive broader sustainability improvements across the built environment, which could have significant implications for Automated National Urban Map Extraction, Scalability of Building Component Data Annotation, General Remote Sensing Image Building Detection, Automated Building Archetype Generation, and Large-Scale Urban Building Function Identification.

This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

📊

Automated Real-World Sustainability Data Generation from Images of Buildings

Peter J Bentley, Soo Ling Lim, Rajat Mathur, Sid Narang

When data on building features is unavailable, the task of determining how to improve that building in terms of carbon emissions becomes infeasible. We show that from only a set of images, a Large Language Model with appropriate prompt engineering and domain knowledge can successfully estimate a range of building features relevant for sustainability calculations. We compare our novel image-to-data method with a ground truth comprising real building data for 47 apartments and achieve accuracy better than a human performing the same task. We also demonstrate that the method can generate tailored recommendations to the owner on how best to improve their properties and discuss methods to scale the approach.

8/29/2024

Automated National Urban Map Extraction

Hasan Nasrallah, Abed Ellatif Samhat, Cristiano Nattero, Ali J. Ghandour

Developing countries usually lack the proper governance means to generate and regularly update a national rooftop map. Using traditional photogrammetry and surveying methods to produce a building map at the federal level is costly and time consuming. Using earth observation and deep learning methods, we can bridge this gap and propose an automated pipeline to fetch such national urban maps. This paper aims to exploit the power of fully convolutional neural networks for multi-class buildings' instance segmentation to leverage high object-wise accuracy results. Buildings' instance segmentation from sub-meter high-resolution satellite images can be achieved with relatively high pixel-wise metric scores. We detail all engineering steps to replicate this work and ensure highly accurate results in dense and slum areas witnessed in regions that lack proper urban planning in the Global South. We applied a case study of the proposed pipeline to Lebanon and successfully produced the first comprehensive national building footprint map with approximately 1 Million units with an 84% accuracy. The proposed architecture relies on advanced augmentation techniques to overcome dataset scarcity, which is often the case in developing countries.

5/6/2024

Scalability in Building Component Data Annotation: Enhancing Facade Material Classification with Synthetic Data

Josie Harrison, Alexander Hollberg, Yinan Yu

Computer vision models trained on Google Street View images can create material cadastres. However, current approaches need manually annotated datasets that are difficult to obtain and often have class imbalance. To address these challenges, this paper fine-tuned a Swin Transformer model on a synthetic dataset generated with DALL-E and compared the performance to a similar manually annotated dataset. Although manual annotation remains the gold standard, the synthetic dataset performance demonstrates a reasonable alternative. The findings will ease annotation needed to develop material cadastres, offering architects insights into opportunities for material reuse, thus contributing to the reduction of demolition waste.

4/15/2024

Generating Synthetic Satellite Imagery for Rare Objects: An Empirical Comparison of Models and Metrics

Tuong Vy Nguyen, Johannes Hoster, Alexander Glaser, Kristian Hildebrand, Felix Biessmann

Generative deep learning architectures can produce realistic, high-resolution fake imagery -- with potentially drastic societal implications. A key question in this context is: How easy is it to generate realistic imagery, in particular for niche domains. The iterative process required to achieve specific image content is difficult to automate and control. Especially for rare classes, it remains difficult to assess fidelity, meaning whether generative approaches produce realistic imagery and alignment, meaning how (well) the generation can be guided by human input. In this work, we present a large-scale empirical evaluation of generative architectures which we fine-tuned to generate synthetic satellite imagery. We focus on nuclear power plants as an example of a rare object category - as there are only around 400 facilities worldwide, this restriction is exemplary for many other scenarios in which training and test data is limited by the restricted number of occurrences of real-world examples. We generate synthetic imagery by conditioning on two kinds of modalities, textual input and image input obtained from a game engine that allows for detailed specification of the building layout. The generated images are assessed by commonly used metrics for automatic evaluation and then compared with human judgement from our conducted user studies to assess their trustworthiness. Our results demonstrate that even for rare objects, generation of authentic synthetic satellite imagery with textual or detailed building layouts is feasible. In line with previous work, we find that automated metrics are often not aligned with human perception -- in fact, we find strong negative correlations between commonly used image quality metrics and human ratings.

9/4/2024