Enhanced Beach Photo Translation using Modified Unsupervised GAN with Regularization

Abstract

To optimize time and cost, tourists often require tools to modify the sky background and atmosphere in beach photos, such as replacing blue-sky views with sunsets or vice versa. Independent modification of the sky and sea is difficult because their color palettes are similar. Another problem that often occurs in image translation is the scarcity of paired datasets, and beach photo datasets are particularly limited in lighting conditions, weather variations, and viewing perspectives. This limitation can cause Generative Adversarial Networks (GAN) models to lose their generalization ability, become prone to overfitting, and produce visual artifacts in the outputs. Therefore, this study proposes an unsupervised GAN approach using a modified CycleGAN and improves its performance for beach image translation by integrating identity mapping, -parameter optimization, a multiscale kernel, and regularization techniques. CycleGAN consists of two generators and two discriminators. The sunset generator translates a blue sky into a sunset sky; the generated output is then passed to the sunset discriminator to determine whether it is real or fake. The generator input image is resized and normalized through preprocessing. The generator architecture is structured to enhance image reconstruction and feature extraction. The details of the translation results are fine-tuned using a 30x30 PatchGAN discriminator and a multiscale kernel convolutional layer. The effect of the hyperparameter , which strikes a balance between cycle consistency, structural preservation, and color fidelity, is also investigated in this work. The findings indicate that while higher  values increase generator loss, they also improve consistency, making it harder to handle dark objects and white clothing. To overcome this issue, regularization techniques, namely photometric augmentation and spectral normalization (SN), together with multiscale kernel convolutional (MSCov), have been applied. Photometric augmentation and MSCov are used to enhance the model's robustness to photographic variations, while SN improves its efficiency and stability. The results of the study show that the proposed method improves image translation accuracy as measured by Mean Squared Error (MSE), Structural Similarity Index (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS).

ABSTRAK: Bagi mengoptimum masa dan kos, pelancong sering memerlukan alat menukar foto latar belakang langit dan suasana pantai, seperti mengganti pemandangan langit biru dengan matahari terbenam atau sebaliknya. Pengubahsuaian bebas langit dan laut sukar dilakukan kerana palet warna langit dan laut adalah serupa. Masalah lain sering berlaku dalam menterjemah imej adalah ketiadaan set data foto berpasangan dan foto pantai sering mengalami kepelbagaian terhad, terutama dari segi pencahayaan, variasi cuaca dan perspektif tontonan. Had ini boleh menyebabkan model Rangkain Generatif Adversari (GAN) ??kehilangan keupayaan generalisasi, terdedah kepada terlebih muat dan penghasilan artifak visual dalam hasil terjemahan. Oleh itu, kajian ini mencadangkan pendekatan GAN tanpa pengawasan menggunakan CycleGAN yang diubah suai bagi meningkatkan prestasi terjemahan imej pantai melalui penyepaduan pemetaan identiti, pengoptimuman parameter , kernel berbilang skala dan menggunakan teknik regularisasi. CycleGAN terdiri daripada dua generator dan dua rangkaian neural diskriminator. Generator matahari terbenam digunakan dalam menterjemah langit biru kepada langit matahari terbenam, kemudian imej terhasil dimajukan kepada diskriminator matahari terbenam bagi menentukan sama ada imej terhasil dikelaskan sebagai imej sebenar atau imej palsu. Imej input generator diubah saiz dan dinormalkan melalui prapemprosesan. Seni bina generator distrukturkan dengan meningkatkan pembinaan semula imej dan pengekstrakan ciri. Butiran hasil terjemahan diperhalusi menggunakan diskriminator PatchGAN 30x30 dan lapisan konvolusi kernel berbilang skala. Kesan hiper parameter  turut dikaji bagi mencapai keseimbangan antara ketekalan kitaran, pemeliharaan struktur, dan kesetiaan warna. Dapatan kajian menunjukkan bahawa walaupun nilai  lebih tinggi ianya meningkatkan kehilangan generator dan konsistensi, menjadikannya lebih sukar dalam mengendali objek gelap dan pakaian putih. Bagi mengatasi isu ini, teknik regularization iaitu pembesaran fotometrik dan normalisasi spektral (SN) bersama Konvulasi Kernel Skala Berbilang (MSCov) telah digunakan. Pembesaran fotometrik dan MSCov dilaksanakan dalam meningkatkan keteguhan model pada variasi fotografi, manakala SN digunakan bagi meningkatkan kecekapan dan kestabilan model. Hasil kajian menunjukkan kaedah ini mampu meningkatkan ketepatan hasil terjemahan imej berdasarkan Ralat Purata Kuasa Dua (MSE), Indeks Persamaan Struktur (SSIM) dan Persamaan Tampalan Perseptual Imej Terpelajar (LPIPS).