[Python] How can I improve this approach for natural background extension in an image using...

I'm working on expanding the background of an image using OpenCV in Python. My current approach involves replicating the border and applying a Gaussian blur to the extended areas to blend them into the original image. The goal is to make the background extension look more natural, especially for images with consistent textures.

Here's the code I'm currently using:

import cv2
import numpy as np

def expand_image_with_smart_blend(image_path, top=50, bottom=50, left=50, right=50):
img = cv2.imread(image_path)
original_h, original_w = img.shape[:2]

expanded_img = cv2.copyMakeBorder(img, top, bottom, left, right, cv2.BORDER_REPLICATE)

blurred_img = expanded_img.copy()

if top > 0:
blurred_img[0:top, :] = cv2.GaussianBlur(expanded_img[0:top, :], (51, 51), 0)

if bottom > 0:
blurred_img[original_h + topriginal_h + top + bottom, :] = cv2.GaussianBlur(expanded_img[original_h + topriginal_h + top + bottom, :], (51, 51), 0)

if left > 0:
blurred_img[:, 0:left] = cv2.GaussianBlur(expanded_img[:, 0:left], (51, 51), 0)

if right > 0:
blurred_img[:, original_w + leftriginal_w + left + right] = cv2.GaussianBlur(expanded_img[:, original_w + leftriginal_w + left + right], (51, 51), 0)

cv2.namedWindow("Smart Blended Expanded Image", cv2.WINDOW_NORMAL)
cv2.namedWindow("Orig Image", cv2.WINDOW_NORMAL)
cv2.imwrite('expanded_smart_blended_image.jpg', blurred_img)
cv2.imshow('Smart Blended Expanded Image', blurred_img)
cv2.imshow("Orig Image", img)
cv2.waitKey(0)
cv2.destroyAllWindows()


expand_image_with_smart_blend('test_img.jpg', top=100, bottom=100, left=100, right=100)


What I've tried: cv2.BORDER_REPLICATE: I use this to replicate the edges of the original image into the newly expanded areas. Gaussian blur: Applied to the expanded regions to soften the transitions between the original image and the new areas.

The issue: The results are somewhat acceptable, but the transitions still don't look as natural as I'd like. In particular:

Over-blurring in some areas makes the background look unrealistic. The replication of the edges doesn't always work well for images with more complex textures.

Original Image Result Image

Question: Is there a more sophisticated approach to expanding the background of an image in OpenCV or other libraries that would give more natural, seamless results? I'm open to approaches involving advanced image processing techniques or machine learning. Any approach with Diffusion Models would also work.

Continue reading...