Hybrid approach for comprehensive recognition of line objects contained in high-density piping and instrumentation diagrams using deep learning and rules

https://doi.org/10.1016/j.compchemeng.2026.109572Get rights and content

Abstract

This study presents a hybrid approach for the automated recognition and classification of line objects in piping and instrumentation diagrams (P&IDs), with the goal of supporting the digital transformation of chemical process design and operation. By integrating Deep Learning (DL) techniques with rule-based methods, the proposed approach extracts flow and signal paths from legacy P&ID images, enabling applications such as process simulation, safety verification, and control logic validation. The approach consists of two stages. In the first stage, all line objects in a P&ID are detected and categorized into lines with special signs and continuous lines. A DL model identifies directional arrows and determines the overall flow structure. In the second stage, the continuous lines are further classified into dimension, extension, and leader lines using the rule-based algorithms, according to their functional characteristics. The method was tested on 30 P&ID sheets from Project A and two from Project B. Initially, the model trained on Project A data achieved precision and recall rates of 95.02% and 93.09%, respectively. On Project B, the performance dropped to 88.92% and 84.76% due to domain shift. After applying transfer learning using the four additional Project B sheets, the performance improved to 95.32% precision and 91.55% recall. These results demonstrate the potential of the proposed approach for accurate and scalable conversion of P&ID data into structured formats, contributing to smart plant design and engineering data integration.

Introduction

Modern engineering, procurement, and construction (EPC) companies increasingly rely on digital piping and instrumentation diagram (P&ID) platforms to support chemical process design, monitoring, and control. These systems manage flow paths, equipment, and interconnections critical to plant operations. Compared to traditional paper-based formats, digital P&IDs offer advantages such as improved safety through predictive simulation, faster design validation, and better data sharing and version control across engineering teams.
However, image- and PDF-based P&IDs are still common in practice, especially during inter-organizational collaboration or when dealing with legacy systems. These unstructured formats are often shared due to confidentiality constraints, highlighting the need for technologies that can convert them into structured digital representations. This digitization process typically involves three steps: (1) detecting objects such as symbols, lines, and text, (2) identifying logical connections, and (3) assigning engineering attributes to create a usable digital diagram for simulation, validation, and smart plant applications.
P&IDs primarily comprise three types of graphical elements: symbols, text, and lines. As illustrated in Fig. 1, pipelines that represent process connectivity and fluid flow are typically depicted as thick continuous lines. Thinner continuous lines are also common, serving various engineering purposes such as annotation, dimensioning, and reference, and are classified into leader lines, dimension lines, and extension lines. Other line types include dotted, dashed, or slashed patterns, which represent signal transmission (e.g., electrical or pneumatic), and arrows that indicate flow direction or highlight critical components.
Among these components, line objects are the most difficult to interpret accurately due to their visual similarity and frequent overlap with other objects. First, line segments often intersect with symbols or text, making segmentation difficult. Additionally, some lines share similar geometric properties but serve different functions, complicating functional classification. For example, pipelines and leader lines may both appear as continuous straight segments but differ significantly in their process roles.
To address these challenges, this study proposes a hybrid method for the comprehensive recognition and functional classification of line objects in image-based P&IDs. The method comprises two primary stages: (1) identifying all line segments and classifying them into continuous lines and lines with special signs through a Deep Learning (DL) model, and (2) applying rule-based logic to further classify continuous lines into dimension, extension, and leader types.
The DL model incorporates arrow detection to infer signal or fluid flow direction, thereby supporting the reconstruction of flow and control pathways. This study makes several contributions to the domain of process diagram digitization and design automation. First, it introduces a method capable of recognizing and classifying nearly all types of line objects in high-density P&IDs. Second, it presents a high-accuracy recognition pipeline that combines DL, rule-based techniques, and classical image processing. Third, it defines a complete labeling schema for training deep neural networks specialized in line recognition. Fourth, it establishes a neutral structured output format suitable for downstream use in digital plant systems, including process simulation, control loop verification, and safety analysis.
The remainder of this paper is organized as follows. Section 2 reviews the related work. Section 3 describes the types of line objects in P&IDs and presents the proposed approach. Section 4 details the technical implementation of line classification, while Section 5 reports the experimental results and analysis. Finally, Section 6 concludes the paper.

Access through your organization

Check access to the full text by signing in through your organization.

Access through your organization

Section snippets

Line detection techniques

Line detection is a fundamental topic in computer vision with applications across various domains including lane detection for autonomous vehicles (Tabelini et al., 2021), crack detection in materials (Liu et al., 2023), pose estimation of robotic arms (Chen et al., 2024), detection of blood vessels or neural structures in medical imaging (Joshi et al., 2024, Ryu et al., 2024, Karageorgos et al., 2024), and table structure recognition in documents (Lee et al., 2022). Traditional techniques such

Line objects in P&IDs

Line objects in P&IDs can be categorized into two types based on their shape, as listed in Table 2: continuous lines and lines containing special signs.
In P&IDs, continuous lines primarily represent pipelines, indicating fluid flow and process connectivity. Primary pipelines are typically depicted as thick, continuous lines. Continuous lines can be further classified into leader, dimension, and extension lines based on their engineering context. Dimension lines provide information such as pipe

Image processing-based

This section provides a detailed explanation of the image-processing techniques applied in this study (Fig. 4 (a-1)) for continuous line detection, short dotted line recognition (Fig. 4(g)), and line-thickness calculation (Fig. 5(a)).

Environment and parameter settings for experimentation

Based on the proposed method, a prototype system for line recognition in image-format P&IDs is implemented. The prototype system was developed using Python 3.9 on the Windows 11 operating system. The prototype system was run in a computing environment with an Intel Core i9-10,900 K CPU and 128 GB of RAM. In the prototype system, the recognition of special line signs and arrows and LETR-based continuous line detection required the training of deep learning models. The hardware and software used

Conclusion

This study presents a hybrid method for the automated recognition and functional classification of line objects in image-format piping and instrumentation diagrams (P&IDs), combining deep learning with rule-based techniques. The proposed approach identifies both continuous lines and those containing special signs such as arrows and dotted patterns, followed by the functional sub-classification of continuous lines into categories including dimension, leader, and extension lines.
To train and

Funding

This research was supported by the Basic Science Research Program (No. NRF-2022R1A2C2005879) through the National Research Foundation of Korea (NRF) funded by the Korean government (MSIT), by the Industrial Technology Alchemist Project Program (No. RS-2024–00419010) funded by the Ministry of Trade, Industry & Energy(MOTIE, Korea), and by the Design Industry Technology Development Program (No. RS-2025–04752997) funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea).

CRediT authorship contribution statement

Yoochan Moon: Writing – original draft, Validation, Software, Methodology, Data curation. Seung-Tae Han: Software, Methodology, Data curation. Ji-Beob Kim: Software, Data curation. Choongsub Yeom: Validation, Resources. Duhwan Mun: Writing – review & editing, Supervision, Methodology, Funding acquisition, Conceptualization.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Duhwan Mun reports financial support was provided by National Research Foundation of Korea. Duhwan Mun reports financial support was provided by Korea Ministry of Trade Industry and Energy. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work

References (39)

  • S.T. Han et al.

    Rule-based continuous line classification using shape and positional relationships between objects in piping and instrumentation diagram

    Expert. Syst. Appl.

    (2024)
  • L. Tabelini et al.

    Keep your eyes on the lane: real-time attention-guided lane detection

  • Y. Chen et al.

    Tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved YOLOv8 and RGB-D information fusion

    Sci. Rep.

    (2024)
  • R.C. Joshi et al.

    VisionDeep-AI: deep learning-based retinal blood vessels segmentation and multi-class classification framework for eye diagnosis

    Biomed. Signal. Process. Control

    (2024)
  • K. Ryu et al.

    Deep learning-based vessel automatic recognition for laparoscopic right hemicolectomy

    Surg. Endosc.

    (2024)
  • G.M. Karageorgos et al.

    Deep learning-based automated pipeline for blood vessel detection and distribution analysis in multiplexed prostate cancer images

    Front. Bioinform.

    (2024)
  • O. Russakovsky et al.

    Imagenet large scale visual recognition challenge

    Int. J. Comput. Vis.

    (2015)
  • N. Carion et al.

    End-to-end object detection with transformers

  • J. Kim et al.

    Object Detection in Aerial Images with Uncertainty-Aware Graph Network

    • Cited by (0)

    View full text
    © 2026 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.