Sayez, Niels ; De Vleeschouwer, Christophe ; Delouille, Veronique ; Bechet, Sabrina ; Lefèvre, Laure

submitted to Astronomy & Astrophysics (2025)

Abstract: Context. Image-to-image translation using generative adversarial networks (GANs) has become a standard approach across numerous scientific domains, including heliophysics research. However, GANs are inherently prone to generating hallucinations in their outputs - a characteristic that, while acceptable in some applications, becomes problematic in physics-bound domains. Given that heliophysics research requires strict adherence to physical events, this tendency to produce artificial artifacts should raise significant concerns about their reliability in scientific applications. Aims. This work aim at measuring the discrepancy between GAN-generated solar images and real observations in two heliophysics applications. Another model producing outputs significantly more faithful to the underlying solar phenomena is proposed and com- pared with a GAN-based solution. Methods. We compare two deep learning models: a generative adversarial network architecture based on Pix2Pix, widely adopted in heliophysics and a model trained in a non-adversarial setup and leveraging input-dependent guidance. Results. GANs consistently fall short of conventional models in physics-constrained applications due to hallucinated features. Ad- ditional conditioning of deep learning networks with physics-based constraints significantly enhances cross-modal image-to-image translation.

Keyword(s): Image-to-image Translation ; Generative Adversarial Networks ; Feature Modulation ; Style Transfer ; Photosphere ; Chromosphere ; Solar corona
Funding: B2 /191/P2/DeepSun