How can I achieve non-linear image stretching from draggable points in SwiftUI using Metal or Core Image?

I'm building a SwiftUI component that lets users dynamically stretch an image by dragging control points, similar to Photoshop's Free Transform tool.

I currently use CIPerspectiveTransform, but it only warps the image rather than stretching it from the dragged points. I have set up draggable points and converted view-space coordinates to image space, yet the deformation behaves like a global warp.

What I Have Tried:

• Applied CIPerspectiveTransform, which doesn’t support localized stretching.
• Implemented draggable points to control image manipulation.
• Transformed coordinates from view space to image space.

Expected Behavior:

• When a user drags a corner or edge point, the image should stretch from that point rather than simply warping the perspective.
• Achieve smooth, real-time rendering of the deformed image.

Below is my current implementation:

SWIFTCopied
import SwiftUI
import CoreImage
import CoreImage.CIFilterBuiltins

struct AdjustableImage: View {
    let uiImage: UIImage
    @State private var topLeading: CGPoint = .zero
    @State private var topTrailing: CGPoint = .zero
    @State private var bottomLeading: CGPoint = .zero
    @State private var bottomTrailing: CGPoint = .zero
    @State private var processedImage: UIImage?
    @State private var lastSize: CGSize = .zero

    var body: some View {
        GeometryReader { geometry in
            ZStack {
                if let processedImage = processedImage {
                    Image(uiImage: processedImage)
                        .resizable()
                        .scaledToFit()
                        .frame(width: geometry.size.width, height: geometry.size.height)
                } else {
                    Color.clear
                }

                DraggablePoint(position: $topLeading, geometry: geometry)
                DraggablePoint(position: $topTrailing, geometry: geometry)
                DraggablePoint(position: $bottomLeading, geometry: geometry)
                DraggablePoint(position: $bottomTrailing, geometry: geometry)
            }
            .onAppear {
                updatePoints(for: geometry.size)
                processImage(size: geometry.size)
            }
            .onChange(of: topLeading) { _ in processImage(size: geometry.size) }
            .onChange(of: topTrailing) { _ in processImage(size: geometry.size) }
            .onChange(of: bottomLeading) { _ in processImage(size: geometry.size) }
            .onChange(of: bottomTrailing) { _ in processImage(size: geometry.size) }
        }
    }

    private func updatePoints(for size: CGSize) {
        guard size != lastSize else { return }
        lastSize = size
        
        topLeading = .zero
        topTrailing = CGPoint(x: size.width, y: 0)
        bottomLeading = CGPoint(x: 0, y: size.height)
        bottomTrailing = CGPoint(x: size.width, y: size.height)
    }

    private func processImage(size: CGSize) {
        guard let inputImage = CIImage(image: uiImage) else { return }
        
        let imageSize = uiImage.size
        let scaleX = imageSize.width / size.width
        let scaleY = imageSize.height / size.height

        let transformedPoints = [
            convertPoint(topLeading, scaleX: scaleX, scaleY: scaleY, viewHeight: size.height),
            convertPoint(topTrailing, scaleX: scaleX, scaleY: scaleY, viewHeight: size.height),
            convertPoint(bottomLeading, scaleX: scaleX, scaleY: scaleY, viewHeight: size.height),
            convertPoint(bottomTrailing, scaleX: scaleX, scaleY: scaleY, viewHeight: size.height)
        ]

        guard let filter = CIFilter(name: "CIPerspectiveTransform") else { return }
        filter.setValue(inputImage, forKey: kCIInputImageKey)
        filter.setValue(transformedPoints[0], forKey: "inputTopLeft")
        filter.setValue(transformedPoints[1], forKey: "inputTopRight")
        filter.setValue(transformedPoints[2], forKey: "inputBottomLeft")
        filter.setValue(transformedPoints[3], forKey: "inputBottomRight")

        guard let outputImage = filter.outputImage else { return }
        let context = CIContext()
        guard let cgImage = context.createCGImage(outputImage, from: outputImage.extent) else { return }
        processedImage = UIImage(cgImage: cgImage)
    }

    private func convertPoint(_ point: CGPoint, scaleX: CGFloat, scaleY: CGFloat, viewHeight: CGFloat) -> CIVector {
        let x = point.x * scaleX
        let y = (viewHeight - point.y) * scaleY
        return CIVector(x: x, y: y)
    }
}

SWIFTCopied
struct DraggablePoint: View {
    @Binding var position: CGPoint
    var geometry: GeometryProxy

    var body: some View {
        Circle()
            .fill(Color.blue)
            .frame(width: 20, height: 20)
            .position(position)
            .gesture(
                DragGesture()
                    .onChanged { value in
                        var newLocation = value.location
                        newLocation.x = max(0, min(newLocation.x, geometry.size.width))
                        newLocation.y = max(0, min(newLocation.y, geometry.size.height))
                        position = newLocation
                    }
            )
    }
}

SWIFTCopied
struct SimpleDemo: View {
    var body: some View {
        if let image = UIImage(named: "imgMusic") {
            AdjustableImage(uiImage: image)
                .frame(width: 300, height: 300)
                .border(Color.gray, width: 1)
        } else {
            Text("Image not found")
        }
    }
}

#Preview {
    SimpleDemo()
}

My Questions:

• How can I modify this approach so the image stretches from the dragged points instead of simply warping its perspective?
• Should I implement Metal shaders for real-time vertex manipulation and deformation? If so, how would I map user interaction to vertex updates?
• Is there a Core Image or pure SwiftUI method that supports non-linear, localized stretching?

Current Result:

Desired Outcome: