Image Manipulation with Shaders — Flutter

Hello, hope you liked my previous articles on shaders this article assumes you have read my previous article or you have basic experience with shaders in flutter, if you haven’t read them here are the links to the articles

Flutter Shaders: An Introduction

Creating Custom Shaders in Flutter: A Step-by-Step Guide

Let’s Start

In this article i will take an existing code from the https://shadertoy.com/ and will adjust the glsl code to work with our flutter app.

I will be using this shader https://www.shadertoy.com/view/mdlSzH code

float inverseLerp(float v, float minValue, float maxValue) {
  return (v - minValue) / (maxValue - minValue);
}

float remap(float v, float inMin, float inMax, float outMin, float outMax) {
  float t = inverseLerp(v, inMin, inMax);
  return mix(outMin, outMax, t);
}


void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
    // Normalized pixel coordinates (from 0 to 1)
    vec2 uv = fragCoord/iResolution.xy;
    
    vec3 col = texture(iChannel0, uv).xyz;
    
    // Ripples
    float distToCenter = length(uv - 0.5);
    float d = sin(distToCenter * 50.0 - iTime * 2.0);
    vec2 dir = normalize(uv - 0.5);
    vec2 rippleCoords = uv + d * dir * 0.06;
    col = texture(iChannel0, rippleCoords).xyz;
    
    // Output to screen
    fragColor = vec4(col, 1.0);
}

The above code takes an image and create a ripple effect on the image

The only function we are going to concentrate mainly is main, we to add few parameter and rename the mainImage function to main

Step 1: Add Imports and define uniforms and out

#include <flutter/runtime_effect.glsl>

out vec4 fragColor;
uniform vec2 uSize;
uniform sampler2D image;
uniform float iTime;

We need to add this code on the top of the file.

Step 2: Rename main function

Now the mainImage function will become

void main() {
  // ....
}

Step 3: Add required variables inside main function

Few variables which we use in every shader program are iResolution and fragCoord , Let’s define those variables

void main() {
    vec2 iResolution = uSize;
    vec2 fragCoord = FlutterFragCoord();
    // ....
    // ....
}

Our final code for shader will look something like below

#include <flutter/runtime_effect.glsl>

out vec4 fragColor;
uniform vec2 uSize;
uniform sampler2D image;
uniform float iTime;


float inverseLerp(float v, float minValue, float maxValue) {
    return (v - minValue) / (maxValue - minValue);
}

float remap(float v, float inMin, float inMax, float outMin, float outMax) {
    float t = inverseLerp(v, inMin, inMax);
    return mix(outMin, outMax, t);
}

void main() {
    vec2 iResolution = uSize;
    vec2 fragCoord = FlutterFragCoord();
    vec2 uv = fragCoord/iResolution.xy;

    vec3 col = texture(image, uv).xyz;

    // Ripples
    float distToCenter = length(uv - 0.5);
    float d = sin(distToCenter * 50.0 - iTime * 2.0);
    vec2 dir = normalize(uv - 0.5);
    vec2 rippleCoords = uv + d * dir * 0.06;
    col = texture(image, rippleCoords).xyz;

    // Output to screen
    fragColor = vec4(col, 1.0);
}

Now Let’s work on the flutter code to read image from assets and pass it to the FragmentShader so that shader program can do its tricks.

Step 4: Let’s create a ShaderPainter class

ShaderPainter class will be similar as we discussed in the previous article only modifications are i updated it so it now take image and float variables in a list so we can use single ShaderPainter class with multiple shaders

import 'dart:ui';
import 'dart:ui' as ui;

import 'package:flutter/material.dart';

class ShaderPainter extends CustomPainter {
  final FragmentShader shader;
  final List<double> uniforms;
  final List<ui.Image?> images;

  ShaderPainter(FragmentShader fragmentShader, this.uniforms, this.images)
      : shader = fragmentShader;

  @override
  void paint(Canvas canvas, Size size) {
    for (var i = 0; i < images.length; i++) {
      shader.setImageSampler(i, images[i]!);
    }
    shader.setFloat(0, size.width);
    shader.setFloat(1, size.height);
    for(var i = 0; i < uniforms.length; i++) {
      shader.setFloat(i + 2, uniforms[i]);
    }

    final paint = Paint();

    paint.shader = shader;
    canvas.drawRect(Offset.zero & size, paint);
  }

  @override
  bool shouldRepaint(covariant CustomPainter oldDelegate) => true;
}

The main changes are we are accepting the list of the images and list of double which are then looped and gave as an input to the shader

for (var i = 0; i < images.length; i++) {
  shader.setImageSampler(i, images[i]!);
}
shader.setFloat(0, size.width);
shader.setFloat(1, size.height);
for(var i = 0; i < uniforms.length; i++) {
  shader.setFloat(i + 2, uniforms[i]);
}

Step 5: Read an Image from Assests

Inorder to read an image from assets we use rootBundle.load to get the image as ByteData then we will use decodeImageFromList to convert the ByteData to ui.Image which can be directly passed to a FragmentShader

Here is the sample code to read an image:

import 'dart:ui' as ui;
// ...

ui.Image? image;

// ...
final imageData = await rootBundle.load('assets/dash.jpg');
image = await decodeImageFromList(imageData.buffer.asUint8List());

Step 6: Pass the image and shader to the ShaderPainter

Now all we need to read the shader .frag file from the assets and load it into FragmentProgram and pass the shader to the class ShaderPainter along the image and the values

Below is the complete StatefulWidget which will read image and shader and pass it to ShaderPainter

In the above code we use Timer which will be used to animate the shader so it will generate great illustrations.

Here is a demo of running shaders in flutter

Here is the example code on github

GitHub - ponnamkarthik/flutter_shaders_image

Conclusion

In conclusion, this article will help one how to modify and integrate a shader from the ShaderToy website into a Flutter application. By using the Flutter, we can also create custom shaders using GLSL code and add various visual effects to our app, such as ripples, pixelation, and more...

Thanks for your time.

Hope you like it, if yes clap & share.