StructSR: Refuse Spurious Details in Real-World Image Super-Resolution

Yachao Li, Dong Liang, Tianyu Ding, Sheng-Jun Huang
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing 211106, China. Microsoft, Washington, USA
AAAI 2025
Paper Code arXiv

Comparison of diffusion-based Real-ISR methods with and without StructSR integration. The original methods generate spurious details in both English letters and Chinese characters. Integration with StructSR significantly reduces these artifacts, resulting in more accurate reconstruction.

Abstract

Diffusion-based models have shown great promise in real-world image super-resolution (Real-ISR), but often generate content with structural errors and spurious texture details due to the empirical priors and illusions of these models. To address this issue, we introduce StructSR, a simple, effective, and plug-and-play method that enhances structural fidelity and suppresses spurious details for diffusion-based Real-ISR. StructSR operates without the need for additional fine-tuning, external model priors, or high-level semantic knowledge. At its core is the Structure-Aware Screening (SAS) mechanism, which identifies the image with the highest structural similarity to the low-resolution (LR) input in the early inference stage, allowing us to leverage it as a historical structure knowledge to suppress the generation of spurious details. By intervening in the diffusion inference process, StructSR seamlessly integrates with existing diffusion-based Real-ISR models. Our experimental results demonstrate that StructSR significantly improves the fidelity of structure and texture, improving the PSNR and SSIM metrics by an average of 5.27% and 9.36% on a synthetic dataset (DIV2K-Val) and 4.13% and 8.64% on two real-world datasets (RealSR and DRealSR) when integrated with four state-of-the-art diffusion-based Real-ISR methods.

Method

In the proposed StructSR, the Structure-Aware Screening (SAS) works in the early inference stage and screens out the structural embedding with the most consistent and clearer structure compared to the LR image. In the later inference stage, the Structure Condition Embedding (SCE) uses the structural embedding to guide noise prediction in conjunction with the LR image. The Image Details Embedding (IDE) inserts the structural embedding into the clean latent image at each timestep according to the degradation degree.

Contributions

  1. We propose StructSR, to fully leverage the temporal reconstructed images during the inference process to enhance the structural fidelity of the diffusion-based Real-ISR methods, without introducing any excess fine-tuning, external models’ prior, or high-level semantic knowledge.
  2. We introduce SAS, SCE, and IDE to interactively update the predicted noise and clean images according to the degradation degree of the LR image during inference, enabling a plug-and-play intervention generation process for diffusion-based Real-ISR methods while suppressing potential spurious structure and texture details.
  3. We demonstrate through experiments that the proposed StructSR consistently improves the structural fidelity of diffusion-based and GAN-based Real-ISR methods.

Results

Qualitative comparisons of different Real-ISR methods. Integration with StructSR significantly reduces spurious details of diffusion-based methods, resulting in high fidelity.
Quantitative comparison with state-of-the-art methods on both synthetic and real-world benchmarks. BSRGAN (ICCV2021), Real-ESRGAN (ICCV2021), FeMaSR(ACM Multimedia2022) ,and LDL (CVPR2022) are GAN-based methods. StableSR (IJCV2024), DiffBIR (arXiv2023), PASD (ECCV2024), and SeeSR (CVPR2024) are diffusion-based methods.

BibTeX

@article{li2025structsr,
  title={StructSR: Refuse Spurious Details in Real-World Image Super-Resolution},
  author={Li, Yachao and Liang, Dong and Ding, Tianyu and Huang, Sheng-Jun},
  journal={arXiv preprint arXiv:2501.05777},
  year={2025}
}