E-Locus - Institutional Repository of the University of Crete

Home Collections Type of Work Post-graduate theses

Post-graduate theses

Current Record: 16 of 6511

[Add to Basket]

Identifier

000462979

Title

A commonsense knowledge-driven framework for part-of relation discovery in images

Alternative Title

Μεθοδολογία για τον εντοπισμό σχέσεων μέρους-όλου σε εικόνες αξιοποιώντας γνώση κοινής λογικής

Author

Κουμούρου, Σωτήρης Θ.

Thesis advisor

Πλεξουσάκης, Δημήτριος

Reviewer

Πάτκος, Θοδωρής
Ευθυμίου, Βασίλης

Abstract

The integration of data-driven techniques with knowledge-based methods has led to many accomplishments in the past few years. While machine learning techniques have proven their potential in diverse domains, they often display weaknesses that symbolic methods can help overcome. This thesis focuses on the development of a framework tailored for the detection of partOf relations in images through the integration of commonsense knowledge. A partOf relation represents the association between an entity and its component or constituent part. The primary objective of this framework is to enhance the precision of relation predictions by incorporating external information derived from problem-agnostic knowledge graphs, specifically ConceptNet. The proposed framework, named PReDeCK, relies on graphs for capturing structured, semantic knowledge about commonsense notions. By conducting symbolic reasoning over the extracted information utilizing the expressive capabilities offered by Answer Set Programming (ASP), PReDeCK can eliminate counter-intuitive conclusions and improve the accuracy of relation predictions. Additionally, this thesis addresses the challenge of identifying errors in the outputs generated by object detection models. We expand PReDeCK to detect potential errors by cross-referencing the model’s results with established real-world knowledge. Moreover, techniques for addressing identified errors are also outlined. We perform experimental evaluations to assess the performance of the framework by using a well-known image dataset, Semantic Pascal-Part, which includes a diverse range of everyday objects and their constituent parts. Throughout the experimental phase, we analyze various framework variations to highlight the critical role of a robust and accurate knowledge domain, as well as the impact of incomplete or erroneous data on the results. The outcomes of these experiments verify that the synergy between visual data and commonsense knowledge leads to a significant improvement in the precision of relation detection. Additionally, the results regarding the error detection task, are quite promising, marking an initial step in quality assurance for object detection models. Overall, this thesis provides insights into a commonsense knowledge-driven framework for discovering partOf relations in images and for identifying potential errors in the outputs generated by an object detection model through the use of symbolic reasoning.

Language

English

Subject

Commonsense knowledge

Computer vision

Neural networks

Neurosymbolic AI

Reasoning

Γνώση κοινής λογικής

Νευρωνικά δίκτυα

Συλλογιστική

Υπολογιστική όραση