Deep learningData Science Deep Learning+5
Image Captioning with Deep Learning
Jun 18, 20246 min readRead
Back
June 27, 2024
Computer Vision
Image deblurring using Generative Adversarial Networks
Pfactorial
Share
Introduction
We all take photos. Sometimes those photos might be blurry .We all want a sharp image without blur. What if AI can convert a blurred image to a sharp image. Yes, there are AI models capable of doing that. One of such models is deblur GAN, which uses Generative Adversarial Neural Networks (GAN) to generate sharp image from a blurred image. We will understand more about how image deblurring works in the following sections.
What is a GAN(Generative Adversarial Neural Networks)
GANs are a type of neural network which consists of two parts ,A generator and a discriminator. During training the generator creates sharp images from a blurred image and the job of discriminator is to identify whether the image created by the generator is a real sharp image or a sharp image created by the discriminator .Throughout the training process the generator learns to outsmart the discriminator by generating images that look like real sharp images. After training the model is able to generate images that look like real images
Deblurring
GANs can be used to deblur images. One of the models that can be used for increasing the image resolution is deblur GAN. Its architecture consists of two parts, a generator network and a discriminator network. The generator network takes blurred images as input and aims to output sharp images. The discriminator’s job is to distinguish between the real sharp image and the sharp image generated by the generator.
When the sharp image generated by the generator reaches the discriminator, the discriminator classifies it into sharp image or the image generated by the generator. Then based on this feedback the generator improves itself to trick the discriminator in believing that the image generated by generator is an original sharp image. At the end of the training process the generator becomes so good that it can generate good sharp images that resembles the original ones.
Challenges
- GAN results can be sometimes unstable and can result in incorrect results.
- Large training data is required for getting correct results from a gan.
- Generalization of blurs. As there are different types of blur .It is required to train GAN on different types of blurs
Use Cases
- Identifying criminals from blurred cctv footages
- Restoring old photos
- Improving the quality of photos in low-light conditions or with motion blur.
Implementing your own Deblurring App
So far, we discussed image deblurring using GANs. Now let’s begin the exciting journey of building our own Image deblurring app.
For implementation we will be using the GAN model deblur-GAN. For using the model we need to clone the repository of deblur-gan using this link
Calling the generator
In the cloned repository we can see the main function which is used to deblur images in the deblur image.py file.
Here our trained generator in the repository generates deblurred images and returns the image. We will be using this function in our app to generate the deblurred images. The function normalizes the image before passing it to our model for returning a deblurred normalized image where values of pixels are between -1 and 1.The deprocess image function converts normalized pixel values between -1 and -1 to the range 0 to 255.
But wait? How the generator is trained
The repository uses three models: a generator, a discriminator, and a combined model. These models form the backbone of our system. The generator attempts to generate clear images from blurry ones, while the discriminator acts like a detective, learning to distinguish real clear images from the ones generated by the generator. The combined model ensures a collaboration between these two. It takes the images created by the generator and sends them to the discriminator for evaluation. The feedback from the discriminator guides the generator to improve , creating more realistic images. We go through a series of loops, or epochs, where the models learn to work together. The generator tries its best to create images that are indistinguishable from the clear images. Meanwhile, the discriminator tries to identify images apart from generated ones. This back-and-forth continues for a number of training cycles. Finally, we save our models by storing their weights. This way, we can use them later without having to go through the training process again. The deblur function uses the trained generator model’s weights.
Building the app
For accessing the model from the user interface we need to serve our model through an api.
Don’t be confused with variables used in the api ,they are added for changing values dynamically in the UI. Those variables are not really important for understanding the api‘s working .Here we are simply taking an image input from the html form and returning the deblurred image using the deblur function.
App
If we know some basic html, css and java script ,we can create a simple User interface by integrating our API.
In our app we need to select the image we want to deblur using the ‘Select Image’ button and we can see the deblurred image after clicking the ‘Deblur Image’ button.
Conclusion
In this blog we explored the process of Image deblurring using GAN models. There are also many other deep learning techniques for image deblurring which you can explore yourself if you are interested. I hope you learned something new from our blog. Happy learning
