Emission reporting has become increasingly compulsory for enterprises around the globe, including Small and Medium Enterprises (SMEs). Available tools–typically leveraging costly sensors, manual audits, and complex software–remain inaccessible to businesses with limited resources. This calls for lightweight, low-cost, and easily accessible technologies to help users monitor their day-to-day carbon footprints and identify inefficiencies. In this preliminary study, we explore the feasibility of using Vision-Language Models (VLMs) for detecting appliances and inferencing their operational context in coffee shop environments using only images. We finetuned a YOLOv8s model using a synthesized dataset containing 169 images each capturing common coffee shop appliances such as refrigerators, coffee machines, etc., to perform appliance detection, achieving an overall Mean Average Precision at Intersection over Union (IoU)=0.50 (mAP50) of 0.865 and Mean Average Precision at IoU=0.95 (mAP50-95) of 0.623 on our test set with the strongest performance observed for coffee machines and toasters, reaching an mAP50 of 0.995 and 0.994 respectively. To infer an appliance’ operational context, we used prompt-based semantic reasoning with a pre-trained transformer Contrastive Language Image Pre-training (CLIP) ViT-B/32, obtaining plausible similarity scores for coffee machines, dishwashers placed next to heat sources, and lighting on during daytime between 0.56 and 0.99. Overall, this vision-language framework demonstrates a feasible first step toward a low-cost, sensor-free, and easily-accessible solution that helps SMEs with emission reporting and identifying effective emission reduction measures.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Towards Sensor-Free Emission Monitoring for SMEs With Vision-Language Appliance Detection and Operational Context Inference

  • Muhammad Azeem,
  • Yining Hu

摘要

Emission reporting has become increasingly compulsory for enterprises around the globe, including Small and Medium Enterprises (SMEs). Available tools–typically leveraging costly sensors, manual audits, and complex software–remain inaccessible to businesses with limited resources. This calls for lightweight, low-cost, and easily accessible technologies to help users monitor their day-to-day carbon footprints and identify inefficiencies. In this preliminary study, we explore the feasibility of using Vision-Language Models (VLMs) for detecting appliances and inferencing their operational context in coffee shop environments using only images. We finetuned a YOLOv8s model using a synthesized dataset containing 169 images each capturing common coffee shop appliances such as refrigerators, coffee machines, etc., to perform appliance detection, achieving an overall Mean Average Precision at Intersection over Union (IoU)=0.50 (mAP50) of 0.865 and Mean Average Precision at IoU=0.95 (mAP50-95) of 0.623 on our test set with the strongest performance observed for coffee machines and toasters, reaching an mAP50 of 0.995 and 0.994 respectively. To infer an appliance’ operational context, we used prompt-based semantic reasoning with a pre-trained transformer Contrastive Language Image Pre-training (CLIP) ViT-B/32, obtaining plausible similarity scores for coffee machines, dishwashers placed next to heat sources, and lighting on during daytime between 0.56 and 0.99. Overall, this vision-language framework demonstrates a feasible first step toward a low-cost, sensor-free, and easily-accessible solution that helps SMEs with emission reporting and identifying effective emission reduction measures.