The new Construction 4.0 paradigm takes advantage of existing technologies. In this scope, the development and application of image-based methods for evaluating and monitoring the state of conservation of buildings has shown significant growth, including support for maintenance plans. Recently, powerful algorithms have been applied to automatically evaluate the state of conservation of buildings using deep learning frameworks, which are utilised as a black-box approach. The large amount of data required for training, the difficulty in generalising, and the lack of parameters to assess the quality of the results often make it difficult for non-experts to evaluate them. For several applications and scenarios, simple and more intuitive image-based approaches can be applied to support building inspections. This paper presents the StainView, which is a fast and reliable method. The method is based on the classification of the mosaic image, computed from a systematic acquisition, and allows one to (i) map stains in facades; (ii) locate critical areas; (iii) identify materials; (iv) characterise colours; and (v) produce detailed and comprehensive maps of results. The method was validated in three identical buildings in Bairro de Alvalade, in Lisbon, Portugal, that present different levels of degradation. The comparison with visual inspection demonstrates that StainView enables the automatic location and mapping of critical areas with high efficiency, proving to be a useful tool for building inspection: differences were of approximately 5% for the facade with the worst and average state of conservation, however, the values deteriorate for the facade under good conditions, reaching the double of percentage. In terms of processing speed, StainView allows a facade mapping that is 8–12 times faster, and this difference tends to grow with the number of evaluated façades.