Reinforced concrete frame structures accumulate crack growth, local spalling and irreversible deformation under cyclic loading, and manual inspection alone stays sensitive to viewing angle and recording completeness. This study proposes a building information modeling (BIM) component-level vision-language-sensor fusion model supported by a digital twin, evaluated on three one-third scale two-story two-bay frames that share geometry and reinforcement but differ in concrete batch and damage path. Images, inspection text, acceleration and strain are aligned to ten beam-column nodes through component identifiers, and a low-parameter quality-aware neuro-fuzzy gate assigns sample-level modal weights from computable quality indicators. In-fold finite element domain-randomized pretraining, component topology propagation and a training-phase physical direction constraint form the digital twin support, with strict outer isolation of every test specimen. Under a leave-one-specimen-out protocol across F1, F2 and F3, in which the held-out frame differs from the training frames only in concrete batch and loading path, the quality-aware gate keeps higher relative performance than fixed-weight fusion under image occlusion, sensor degradation and modality absence. Ablation shows that finite element pretraining, component topology and the direction constraint carry distinct diagnostic roles within a three-specimen feasibility range.