Coder Social home page Coder Social logo

appspell / shaderview Goto Github PK

View Code? Open in Web Editor NEW
84.0 4.0 13.0 2 MB

ShaderView is an Android View that makes it easy to use GLSL shaders for your app. It's the modern way to use shaders for Android instead of RenderScript.

License: MIT License

Kotlin 98.95% GLSL 1.05%
android shaders gltextureview textureview androidshader android-opengl customview glsl-shaders glsl shaderview

shaderview's Introduction

ShaderView

This library is the easiest way to use OpenGL shaders as an Android View. You just simply need to add ShaderView in your layout and set up shaders. The advantage of this library that you can use ShaderView in your hierarchy as a regular View.

Use cases:

  • Shaders for video
  • Advanced UI components (blur, shadow, lighting, etc.)
  • UI effects and animation
  • Realtime image animation
  • Shaders for a camera

Table of content

How to use it

Add dependency to the project

Gradle

allprojects {
	repositories {
		...
		maven { url 'https://jitpack.io' }
	}
}
implementation 'com.github.appspell:ShaderView:[last-version]'

Add ShaderView to XML layout

  1. Add ShaderView to the XML layout
    <com.appspell.shaderview.ShaderView
        android:id="@+id/shaderView"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:fragment_shader_raw_res_id="@raw/fragment_shader" />
  1. Set your fragment and vertex (if needed) shaders using the following attributes:

app:fragment_shader_raw_res_id - reference to the fragment shader file in RAW resource solder example

app:vertex_shader_raw_res_id - reference to the vertex shader file in RAW resource solder example

Add ShaderView programmatically (or configure programmatically)

val shaderView = ShaderView(this)

with(shaderView) {
   fragmentShaderRawResId = R.raw.color_frag
   shaderParams = ShaderParams.Builder()
                .addColor("diffuseColor", R.color.teal_200, resources)
                .build()
}

The full list of ShaderView properties:

fragmentShaderRawResId - reference to the vertex shader file in RAW resource solder [example]

vertexShaderRawResId - reference to the fragment shader file in RAW resource solder [example]

shaderParams - custom parameters that we're going to send to the shader (uniform)

onViewReadyListener - called when the view is created and ready to create a shader

onDrawFrameListener - called each frame

updateContinuously - should we render the view each frame (default is "false")

debugMode - enable or disable debug logs

How to send custom data to the shader

Pass ShaderParams to the ShaderView if you need to set up some uniform attributes.

shaderView.shaderParams = ShaderParamsBuilder()
                    .addTexture2D(
                        "uNormalTexture", // name of `sampler2D` in the fragment shader
                        R.drawable.normal_button, // drawable that we use for such texture
                        GLES30.GL_TEXTURE0 // texture slot
                    )
                    .addColor("uColor", R.color.grey, resources) // send color as `uniform vec4`
                    .addVec4f("uColor2", floatArrayOf(0.5f, 0.5f, 0.5f, 1f))
                    .addVec3f("uVaryingColor", floatArrayOf(0.5f, 0.5f, 0.5f))
                    .addFloat("uTime", 1.0f)
                    .build()

During execution, you may update this param:

shaderParams.updateValue("time", System.currentTimeMillis())

If you need to update uniform each frame, you may use onDrawFrameListener.

shaderView.onDrawFrameListener = { shaderParams ->
                    shaderParams.updateValue("time", System.currentTimeMillis())
                }

The full list of supported uniform types: float, int, bool, vec2f, vec3f, vec4f, vec2i, vec3i, vec4i, mat3, mat4, mat3x4, sampler2D, samplerExternalOES

How to add custom fragment shader using build-in vector shader

  1. Set up version
  2. Configure input and output. By default vertex shader sends texture coordinates using this field in vec2 textureCoord
  3. add main() function and return the result color to fragColor
#version 300 es
precision mediump float;

in vec2 textureCoord;
out vec4 fragColor;

void main() {
    fragColor = vec4(textureCoord.x, textureCoord.y, textureCoord.y, 1.0);
}

How to add shaders for video playback

Full code of example using ExoPlayer you may find here and here

  1. Setup OES texture in fragment shader:
#version 300 es
#extension GL_OES_EGL_image_external_essl3 : require

uniform samplerExternalOES uVideoTexture;
  1. Define it for ShaderParams
shaderParams = ShaderParamsBuilder()
                .addTextureOES("uVideoTexture") // video texture input/output
                .build()
  1. When ShaderView is ready, send Surface to the video player
shaderView.onViewReadyListener = { shader ->
                // get surface from shader params
                val surface = shader.params.getTexture2dOESSurface("uVideoTexture")

                // initialize video player with this surface
                initVideoPlayer(surface)
            }

Example of shaders

In Android Demo Project code you may found it in ViewHolders here

Additional information

Why we use TextureView instead of SurfaceView you can read here.

To be able to use OpenGL rendering for Android TextureView, we've created GLTextureView.kt

shaderview's People

Contributors

appspell avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

shaderview's Issues

Strange behavior: doesn't work in real project

Everything works in an empty project, but does not work when trying to add ShaderView to a real project. My goal is simple - display a bitmap with shader effects and dynamically change it. But for some reason, I can't even display the bitmap.

My layout:

  <com.appspell.shaderview.ShaderView
      android:id="@+id/shaderView"
      android:layout_width="match_parent"
      android:layout_height="match_parent" />

My code:

// Called when bitmap loaded
binding.shaderView.apply {
            fragmentShaderRawResId = R.raw.fragment_texture_shader
            shaderParams = ShaderParamsBuilder()
                .addTexture2D("uTexture", bitmap, GLES30.GL_TEXTURE0)
                .build()
        }

My shader:

#version 300 es
precision mediump float;

uniform sampler2D uTexture;

in vec2 textureCoord;
out vec4 fragColor;

void main() {
   fragColor = texture(uTexture, textureCoord);
}

Result - white ShaderView.

I tried:

  • Set updateContinuously = true
  • Set debugMode = true (no logs messages at all)
  • Use drawable instead bitmap
  • Create ShaderView programmatically
  • Use shaderView.shaderParams.updateValue2D()
  • Set in manifest android:hardwareAccelerated="true"

Nothing works. Maybe there are some unobvious restrictions? The best that I managed to achieve was to display drawable (png) if ShaderView is initialized immediately in onCreate.

I need your ideas how to simplify interface!

I want to make the simpler way how to use this library.
Do you have any ideas on how to make the architecture or API library more friendly?

Look at the code below, seems like there is a better (simpler) way to set up this view. Maybe some DSL?

shaderView.apply {
                updateContinuously = true // update each frame
                vertexShaderRawResId = R.raw.quad_tangent_space_vert
                fragmentShaderRawResId = R.raw.nomral_map
                shaderParams = ShaderParamsBuilder()
                    .addTexture2D(
                        "uNormalTexture",
                        R.drawable.normal_button,
                        GLES30.GL_TEXTURE0
                    )
                    .addColor("uColor", R.color.grey, resources)
                    .addVec3f("uVaryingColor", floatArrayOf(0.5f, 0.5f, 0.5f))
                    .addVec3f("uLightDirection", floatArrayOf(1.0f, 1.0f, 0.0f))
                    .addVec3f("uEyeDirection", floatArrayOf(0.0f, 0.0f, 0.0f))
                    .build()
                onDrawFrameListener = { shaderParams ->
                    val pos = (System.currentTimeMillis() % 5000L) / 1000f
                    shaderParams.updateValue("uLightDirection", floatArrayOf(0.0f + pos, 1.0f, 0.0f))
                }
            }

Performance issue (not really)

Firstly,
thanks for handy tool, really useful to avoid messing directly with OGL.

I have a question or an issue to rise though.

I've just been testing it little bit and did following simple fragmentShader

#version 300 es

#ifdef GL_ES
precision mediump float;
#endif

uniform float u_time;
uniform vec2 u_resolution;

out vec4 fragColor;

float nsin(float value) {
    return (sin(value) + 1.) / 2.;
}

void main() {
    vec2 uv = (gl_FragCoord.xy - 0.5 * u_resolution.xy) / u_resolution.y;
    uv *= 2.0;

    float x = length(uv) * 10.0 * nsin(3. * u_time);
    fragColor = vec4(vec3(x), 1.0) + vec4(1.0, 0.0, 0.0, 0.1);
}

^ simple animation put in color_frag.sh

and updated the SimpleShaderActivity
with

binding.shaderView.updateContinuously = true
        binding.shaderView.debugMode = true

        binding.shaderView.onDrawFrameListener = { params ->
            if (startTime == 0f) {
                startTime = now()
            }
            params.updateValue("u_time", now())
            params.updateValue("u_resolution", viewSize)
        }

It works fine on start, but in time let's say in a min or 2,
the performance goes really down and FPS very noticable drops to something like 10 or so, it looks like it's dropping with running time.

Any idea if I'm doing something wrong or where might be an issue ?

Surface is invalid when put ShaderView in ViewPager2 and swipe to previous page item

Assume there are two Fragment (AFragment and BFragment) in ViewPager2.
Each Fragment contains one ShaderView (AShaderView and BShaderView).

When we swipe the ViewPager2:

AFragment -> BFragment -> AFragment

The Surface is invalid that used by AShaderView.


My survey:

The first step AFragment -> BFragment cause the Surface in AShaderView been release in

override fun release() {
for (key in map.keys) {
map[key]?.apply {
when (valeType) {
Param.ValueType.SAMPLER_OES -> {
(value as? TextureOESParam)?.apply {
surfaceTexture.release()
surface.release()
}
}
else -> {
// do nothing
}
}
}
}
}

(Called from onSurfaceTextureDestroyed)

and the second step BFragment -> AFragment using the Surface which had been release in the previous step.

It can not create a new Surface because Param.ValueType.SAMPLER_OES value is not null

Param.ValueType.SAMPLER_OES -> {
if (value == null) {
// if it's not initialized
location = createExternalTexture()
val surfaceTexture = SurfaceTexture(location)
value = TextureOESParam(
surfaceTexture = surfaceTexture,
surface = Surface(surfaceTexture)
).apply {
surfaceTexture.setOnFrameAvailableListener {
lock.withLock {
updateSurface.set(true)
}
}
}
}
}

Throws an exception if the same bitmap is passed multiple times

I've put this code in the onBindViewHolder function in a recycler view, and it keeps throwing exceptions about the bitmap being recycled.

shaderView.shaderParams?.updateParam(
    "albedo", Param(
        Param.ValueType.SAMPLER_2D,
        value = TextureParam(
            bitmap = lastCoverBitmap,
            textureSlot = GLES30.GL_TEXTURE0
        )
    )
)

The exception:

java.lang.IllegalArgumentException: bitmap is recycled
    at android.opengl.GLUtils.texImage2D(GLUtils.java:152)
    at com.appspell.shaderview.ext.TextureKt.toGlTexture(Texture.kt:66)
    at com.appspell.shaderview.gl.params.ShaderParamsImpl.bindTextures(ShaderParamsImpl.kt:153)
    at com.appspell.shaderview.gl.params.ShaderParamsImpl.bindParams(ShaderParamsImpl.kt:74)
    at com.appspell.shaderview.gl.shader.GLShaderImpl.bindParams(GLShaderImpl.kt:89)
    at com.appspell.shaderview.ShaderView.initShaders(ShaderView.kt:177)
    at com.appspell.shaderview.ShaderView.access$initShaders(ShaderView.kt:28)
    at com.appspell.shaderview.ShaderView$rendererListener$1.onSurfaceCreated(ShaderView.kt:92)
    at com.appspell.shaderview.gl.render.GLQuadRenderImpl.onSurfaceCreated(GLQuadRender.kt:90)
    at com.appspell.shaderview.gl.view.GLTextureView$GLThread.guardedRun(GLTextureView.kt:1507)
    at com.appspell.shaderview.gl.view.GLTextureView$GLThread.run(GLTextureView.kt:1223)

I'm not sure yet, but maybe it's caused by bitmap?.toGlTexture(needToRecycle = true, textureParam.textureSlot) in the ShaderParamsImpl.bindTextures function
Maybe I'll try to fix it myself, but can't promise anything

UPDATE:
I forked the repo and tried to fix it, but have no idea of how to test it, cause it seems incompatible with jitpack, and jfrog/jcenter seems like it's a paid service, idk

Should I open a pull request, even though I haven't tested it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.