Replicating CSS Object-Fit in WebGL: Optimized Techniques for Image Scaling and Positioning

If you’ve ever worked with images on the web, you’re probably familiar with the CSS property object-fit: cover. This property ensures that an image completely covers its container while preserving its aspect ratio, which is crucial for responsive layouts. However, when it comes to WebGL, replicating this effect isn’t as straightforward. Unlike traditional 2D images, WebGL involves applying textures to 3D meshes, and this brings with it a set of performance challenges.

In this article, we’ll explore several techniques to achieve the object-fit: cover; effect in WebGL. I first came across a particularly elegant method in this article on Codrops, written by Luis Bizarro (thank you, Luis!). His approach cleverly calculates the aspect ratio directly within the fragment shader:

// https://tympanus.net/codrops/2021/01/05/creating-an-infinite-auto-scrolling-gallery-using-webgl-with-ogl-and-glsl-shaders/

precision highp float;
 
uniform vec2 uImageSizes;
uniform vec2 uPlaneSizes;
uniform sampler2D tMap;
 
varying vec2 vUv;
 
void main() {
  vec2 ratio = vec2(
    min((uPlaneSizes.x / uPlaneSizes.y) / (uImageSizes.x / uImageSizes.y), 1.0),
    min((uPlaneSizes.y / uPlaneSizes.x) / (uImageSizes.y / uImageSizes.x), 1.0)
  );
 
  vec2 uv = vec2(
    vUv.x * ratio.x + (1.0 - ratio.x) * 0.5,
    vUv.y * ratio.y + (1.0 - ratio.y) * 0.5
  );
 
  gl_FragColor.rgb = texture2D(tMap, uv).rgb;
  gl_FragColor.a = 1.0;
}

I won’t go into the specifics of setting up a 3D environment here, as the focus is on the logic and techniques for image scaling and positioning. The code I’ll provide is intended to explain the core concept, which can then be adapted to various WebGL setups.

The approach we’re using involves calculating the aspect ratios of both the plane and the image, then adjusting the UV coordinates by multiplying these ratios to proportionally zoom in the image, ensuring it doesn’t go below a scale of 1.0.

While this technique is effective, it can be computationally expensive because the calculations are performed for each fragment of the mesh and image continuously. However, in our case, we can optimize this by calculating these values once and passing them to the shader as uniforms. This method allows us to maintain full control over scaling and positioning while significantly reducing computational overhead.

That said, the shader-based approach remains highly useful, particularly in scenarios where dynamic recalculation is required.

Main Logic: Calculating Image and Mesh Ratios

Now let’s move on to the main logic, which allows us to calculate whether our mesh is larger or smaller than our image and then adjust accordingly. After defining the size of our mesh as described above, we need to calculate its ratio, as well as the ratio of our image.

If the ratio of our image is larger than the ratio of our mesh, it means the image is wider, so we must adjust the width by multiplying it by the ratio between the image’s ratio and the mesh’s ratio. This ensures that the image completely covers the mesh in width without distortion, while the height remains unchanged.

Similarly, if the ratio of our image is smaller than that of our mesh, we adjust the height to extend it to completely fill the height of the mesh.

If both conditions are met, it means that the ratio is the same between the two.

fitImage(){
    const imageRatio = image.width / image.height;
    const meshSizesAspect = mesh.scale.x / mesh.scale.y;

    let scaleWidth, scaleHeight;

    if (imageRatio > meshSizesAspect) {
        scaleWidth = imageRatio / meshSizesAspect;
        scaleHeight = 1;
    } else if (imageRatio < meshSizesAspect) {
        scaleWidth = 1;
        scaleHeight = meshSizesAspect / imageRatio;
    } else {
        scaleWidth = 1;
        scaleHeight = 1;
    }

    this.program.uniforms.uPlaneSizes.value = [scaleWidth, scaleHeight];
}

After calculating this, we now need to adjust it with our UVs.

// vertex.glsl

precision highp float;

attribute vec3 position;
attribute vec2 uv;

uniform mat4 modelViewMatrix;
uniform mat4 projectionMatrix;

uniform vec2 uImagePosition;
uniform vec2 uPlaneSizes;

varying vec2 vUv;

void main() {
    gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
    vUv = uImagePosition + .5 + (uv - .5) / uPlaneSizes;
}

// fragment.glsl

precision highp float;
varying vec2 vUv;
uniform sampler2D tMap;

void main() {
    gl_FragColor = texture2D(tMap, vUv);
}

So, after normalizing the UVs, we divide them by the vec2 that we calculated previously to adjust the UVs so that the image is mapped correctly onto the mesh while respecting the proportions of uPlaneSizes.

Now, we can simply compute the values of uPlaneSizes within a function to initially calculate them on each resize. This is also useful when we want to deform the dimensions of our mesh, for example, if we want to close an image while maintaining a visually aesthetic ratio.

Animation Examples

Example 1: Basic Scaling Without fitImage()

const start = {
    x: this.mesh.scale.x,
}

new Update({
    duration: 1500,
    ease: [0.75, 0.30, 0.20, 1],
    update: (t) => {
        this.mesh.scale.x = Lerp(start.x, start.x * 2, t.ease);
    },
}).start();

Example 2: Integrating fitImage() in the Update Loop

By recalculating ratios during animations, we ensure consistent scaling:

const start = {
    x: this.mesh.scale.x,
}

new Update({
    duration: 1500,
    ease: [0.75, 0.30, 0.20, 1],
    update: (t) => {
        this.mesh.scale.x = Lerp(start.x, start.x * 2, t.ease);
        this.fitImage(); // Execute the method only during the animation
    },
}).start();

Vertical Translation with uImagePosition

Modifying the uImagePosition uniform allows for offsetting the texture coordinates, mimicking a div with overflow: hidden:

new Update({
    duration: 1500,
    ease: [0.1, 0.7, 0.2, 1],
    update: (t) => {
        this.program.uniforms.uImagePosition.value[1] = Lerp(0, 1, t.ease);
    },
}).start();

This shifts the image vertically within the mesh, similar to translating a background image.

Combining Mesh Scaling and Internal Positioning

To animate a mesh closing vertically while maintaining alignment (like transform-origin: top in CSS):

const start = {
    y: this.mesh.scale.y,
    x: this.mesh.scale.x,
    position: this.mesh.position.y,
}

new Update({
    duration: 1500,
    ease: [0.1, 0.7, 0.2, 1],
    update: (t) => {
        const ease = t.ease;
        this.mesh.scale.y = Lerp(start.y, 0, ease);
        this.program.uniforms.uImagePosition.value[1] = Lerp(0, .25, ease);
        this.fitImage();
    },
}).start();

Adjusting Transform Origin

By default, the mesh will shrink toward the center. To keep it anchored to the top (equivalent to CSS’s transform-origin: top), adjust the mesh’s position by half the scale value during transformations:

// Add this line in the update method
this.mesh.position.y = Lerp(start.position, start.y / 2, ease);

Combining Mesh Scaling and Uniform Adjustments

By modifying both the mesh size and uniforms, we can create interesting visual effects. For example, here’s an idea for a vertical slider with two images:

Thank you for reading! I hope this guide helps you optimize image manipulation in WebGL. If you have any questions, suggestions, or just want to share your thoughts, feel free to reach out.