Visual analysis of printed circuit boards

Abstract

Sophisticated object detection is used in automated systems throughout many areas of application. This work elaborates the possibilities of utilizing a modern object detection approach for visual analysis of printed circuit boards (PCB) in the context of hardware-related security assessments. In detail, the task is to reliably localize and classify different types of integrated circuits and other components on image data depicting electronic devices. Faster R-CNN, an object detection architecture based on modern convolutional neural networks, is selected for this purpose. In order to effectively train the detector a variety of datasets are collated that contain suitable annotations for PCB com- ponents. In parallel, an internal image acquisition process further complements the available datasets. All dataset are extended by introducing six new sub-categories for integrated circuits. This approach of IC class breakdown is unique and not present in any of the investigated datasets so far. In addition, four new categories for passive and miscellaneous components are also introduced into the data. Finally, the object detection architecture is appropriately configured and several models are trained with different combinations of the available image data. The best model yields an overall detection performance of 0.57 mAP. Detection scores of individual classes tend to be more vulnerable to small component sizes and high intra-class variance than to class imbalances of the datasets. Acceptable performance scores above 0.6 AP are reported for four out of ten classes, two even reach 0.75 AP.