How can I identify and group similar images using GPT-4 Vision without metadata?

You can separate common images (i.e. group visually similar images) without relying on metadata by comparing their visual content. Here are two common approaches:

---

1. Perceptual Hashing

Perceptual hashing (pHash) generates a hash for each image based on its visual appearance. Images that are visually similar tend to have similar hash values. You can then compare these hashes (using, for example, the Hamming distance) to cluster similar images.

Example using Python with the imagehash library:

PYTHONCopied
from PIL import Image
import imagehash
import os

# Compute pHash for each image and store in a dictionary
image_folder = 'path/to/your/images'
hash_dict = {}

for filename in os.listdir(image_folder):
    if filename.lower().endswith(('.png', '.jpg', '.jpeg', '.bmp', '.gif')):
        file_path = os.path.join(image_folder, filename)
        img = Image.open(file_path)
        # Compute a perceptual hash
        phash = imagehash.phash(img)
        hash_dict.setdefault(str(phash), []).append(filename)

# Output groups of similar images (exact hash match finds identical pHashes)
for phash, files in hash_dict.items():
    if len(files) > 1:
        print(f"Group with hash {phash}: {files}")

Note: In practice, due to minor differences (e.g. compression), you might want to allow a small Hamming distance threshold rather than an exact match. You can compare hashes pairwise and group those that are within your chosen threshold.

---

2. Feature Extraction with Deep Learning

Another approach is to extract deep features from the images (using a pretrained CNN like ResNet, VGG, or Inception) and then perform clustering (e.g. k-means or DBSCAN) on those high-dimensional feature vectors.

Example using Keras and scikit-learn:

PYTHONCopied
import os
import numpy as np
from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input
from tensorflow.keras.preprocessing import image
from sklearn.cluster import DBSCAN

# Initialize the pre-trained model (exclude the top fully connected layers)
model = ResNet50(weights='imagenet', include_top=False, pooling='avg')

def extract_features(img_path, target_size=(224, 224)):
    img = image.load_img(img_path, target_size=target_size)
    img_array = image.img_to_array(img)
    img_array = np.expand_dims(img_array, axis=0)
    img_array = preprocess_input(img_array)
    features = model.predict(img_array)
    return features.flatten()

# Extract features for all images
image_folder = 'path/to/your/images'
features_list = []
filenames = []

for fname in os.listdir(image_folder):
    if fname.lower().endswith(('.png', '.jpg', '.jpeg', '.bmp', '.gif')):
        full_path = os.path.join(image_folder, fname)
        features = extract_features(full_path)
        features_list.append(features)
        filenames.append(fname)

features_array = np.array(features_list)

# Cluster using DBSCAN which doesn't require specifying the number of clusters
clustering = DBSCAN(eps=0.5, min_samples=2, metric='euclidean').fit(features_array)

# Group images by cluster label
clusters = {}
for label, fname in zip(clustering.labels_, filenames):
    clusters.setdefault(label, []).append(fname)

for label, group in clusters.items():
    if label != -1:  # -1 is generally noise
        print(f"Cluster {label}: {group}")

Tips:

• You may need to adjust parameters (like eps for DBSCAN) to get the desired clustering result.
• Experimenting with different models or layers (features from earlier layers might capture more local details) can improve grouping results.

---

Conclusion

Both methods work without relying on metadata. Perceptual hashing is usually simpler and faster for detecting near duplicates, while deep feature extraction followed by clustering can help separate images based on more complex visual patterns. Choose the approach based on your specific needs and the variability in your dataset.