Contents 1 GLESGAE - Shader Based Transformations 1.1 Introduction 1.2 Fast Track 1.3 Meet the... lack of Stacks? 1.4 Shader Uniform Management 1.5 Wait, what Camera?! 1.6 The MVP Uniform 1.7 Cameras are More Fun 1.8 Matrix Bashing 1.9 Building the Example 2 Next Time

GLESGAE - Shader Based Transformations

Introduction

Whereas most Fixed Function vs Shader Based stuff does go off in different directions, I'm going to have to ask you to read the Fixed Function Transformations article first: GLESGAE:Fixed Function Transformations

Reason being, we went through some of the Camera code, and the whole Right Vector issue... both of which are still relevant; the right vector issue remaining until we sort out a proper mesh format - which'll happen when we switch over to VBOs.

Fast Track

We're be moving to SVN revision 6... svn co -r 6 http://svn3.xp-dev.com/svn/glesgae/trunk/ glesgae

Meet the... lack of Stacks?

ES 2.0 has no Matrix Stacks.
You don't have to deal with switching matrix mode, and pushing and popping anything on a stack - they don't exist.
You're expected to calculate the final matrix and send it over to a shader for processing, and this is where the ubiquitous mvp transform comes in - the ModelViewProjection big daddy matrix.

Shaders are good in that they offload from the CPU to the GPU, but there are still times the CPU is king at performing certain tasks; or indeed in this case, is the only thing that can access the memory required. If we were to send over the three matrices manually, and have the GPU calculate them in whatever variants it needed, it'd actually be slower than doing it once on the CPU and sending it over in the pre-built manner. In this case it comes down to the sending of three matrices rather than one. Such is the balancing act you have to perform when dealing with shader based rendering.

But first, we need to write a uniform system....

Shader Uniform Management

As we've gone over before when writing the initial Shader Render Context, the Shader is king of the renderer.
It describes what attributes we're going to use, as well as what it does to them before they get to the screen.
Uniforms are used to give extra information from the engine to the shader, so that we can do more things with it.

There are several ways in which we can deal with this, but my preferred method is to have a bunch of updaters for each uniform we're interested in. This way, we keep code duplication down, and we decouple what's actually in the shaders from the engine. We can go all out and have funky caching mechanisms to ensure we only update when we really need to... but for us, we'll just go with a simple system that each shader will run through a map of updaters to update their own uniforms. Keeps things nice and straight forward. You can also learn more about a method such as this from Game Engine Gems 2, if you're interested, which also describes deferred OpenGL calls amongst other handy gems of information.

So, let's create our Shader Uniform Updater interface:

#ifndef _SHADER_UNIFORM_UPDATER_H_
#define _SHADER_UNIFORM_UPDATER_H_

#if defined(GLX)
	#include "GLee.h"
#elif defined(GLES1)
	#if defined(PANDORA)
		#include <GLES/gl.h>
	#endif
#elif defined(GLES2)
	#if defined(PANDORA)
		#include <GLES2/gl2.h>
	#endif
#endif

namespace GLESGAE
{
	class Camera;
	class Material;
	class Matrix4;
	class ShaderUniformUpdater
	{
		public:
			virtual ~ShaderUniformUpdater() {}
			
			/// Pure virtual to ensure you overload the update function!
			virtual void update(const GLint uniformId, const Camera* const camera, const Material* const material, const Matrix4* const transform) = 0;
			
		protected:
			/// Protected constructor to ensure you derive from this, and don't create empty updaters.
			ShaderUniformUpdater() {}
	};
}

#endif

Another nice and simple class. See a pattern here? Engine code should be simple and get out the way, so that you can extend it and do what you need when writing your application or game!
Anyway... only partial oddity here is that we do define an include for GLES1 as a "just in case" measure. If the header gets pulled in by accident, it'll still compile. We're not doing anything particularly fancy here anyway, as we just define an interface method that takes in a GLint, Camera, Material and Transform.

We then add a map to the Shader Based Render Context, and a couple of functions:

public:
/// Add a uniform updater
void addUniformUpdater(const std::string& uniformName, ShaderUniformUpdater* const updater);

/// Clear uniform updaters
void clearUniformUpdaters() { mUniformUpdaters.clear(); }

protected:
/// Update all uniforms
void updateUniforms(Material* const material, Matrix4* const transform);

private:
std::map<std::string, ShaderUniformUpdater*> mUniformUpdaters;

Obviously stick them in the right parts of the file.
We make the add and clear functions public so that we can access them directly, and protect the updateUniforms call as we'll be handling this in-class. Of course, our actual map itself is private, out the way of meddling hands.

The addUniformUpdater call does exactly what you think it does.. it just adds the updater to the map with the uniformName as the key. Again, I'm being lazy and not checking anything, but you really should... and we shall have a clean-up day soon which'll involve writing a logging system and error checking macros. Makes things easier when certain platforms log things in peculiar ways, or when asserting could actually destroy the call stack before you get to see it!

The updateUniforms function is relatively straight forward too.. I'll print it up here though and we can go through it:

void ShaderBasedContext::updateUniforms(Material* const material, Matrix4* const transform)
{
	std::vector<std::pair<std::string, GLint> > uniforms(mCurrentShader->getUniformArray());
	for (std::vector<std::pair<std::string, GLint> >::iterator itr(uniforms.begin()); itr < uniforms.end(); ++itr)
		mUniformUpdaters[itr->first]->update(itr->second, mCamera, material, transform);
}

We take the material and transform in because we don't actually store these as member data. We do store the Camera as member data, however, so we can access this from wherever we like. Again, naughty points for not checking pointers.. you should! Same for ensuring that the uniform updater we're looking for actually exists.
But in essence, all we do here is run through the uniform array on our current shader, match up with an updater, and call it's update function.
And we call this function just after our bindShader call in the drawMesh function.

Wait, what Camera?!

I've mentioned storing the Camera as member data, except we haven't written any code to pass it through yet!
We shall do that now.

public:
/// Set the Camera
void setShaderBasedCamera(Camera* const camera) { mCamera = camera; }

private:
Camera* mCamera;

Done!

What, you were expecting something a bit more substantial like in the Fixed Function Context?
Remember, in a Shader Based Context, you have to do everything through shaders.. your entire rendering pipeline is flexible and up to you to decide what to do with it. There are no matrix stacks to deal with, no fixed light count, and no easy way out. If you want to render something, you push it through a shader, and feed that shader with enough information to make it do what you want... which in our case will be the Camera matrices which we had pushed back on to the Projection and ModelView stacks last time.

Of course, you'll need to set the Graphics System to call the correct setCamera function, but this is trivial, and in the repository for the curious.

The MVP Uniform

Now we need to send our Camera matrices to our shader.
As we've mentioned, we'll precompute this so we don't have to send three matrices over all the time.

#ifndef _MVP_UNIFORM_UPDATER_H_
#define _MVP_UNIFORM_UPDATER_H_

#include "../../Graphics/ShaderUniformUpdater.h"

class MVPUniformUpdater : public GLESGAE::ShaderUniformUpdater
{
	public:
		MVPUniformUpdater() : GLESGAE::ShaderUniformUpdater() {}
		
		void update(const GLint uniformId, const GLESGAE::Camera* const camera, const GLESGAE::Material* const material, const GLESGAE::Matrix4* const transform);
};

#endif

You might wonder where on earth this file is, considering the include definition is a bit mad.
This updater lives in the example folder - away from the core engine. Uniform updaters are generally app specific, so we store them outside the engine. Granted, a ModelViewProjection updater is probably going to be the same for every application, but note the "probably" .. if we didn't have the ability to change it, we'd likely need to hack around it if we needed to!

Anyway, our object file is nice and simple, as it just contains the one method:

#include "MVPUniformUpdater.h"

#if defined(GLX)
	#include "../../Graphics/GLee.h"
#elif defined(GLES2)
	#if defined(PANDORA)
		#include <GLES2/gl2.h>
	#endif
#endif

#include "../../Maths/Matrix4.h"
#include "../../Graphics/Camera.h"
#include "../../Graphics/Material.h"

using namespace GLESGAE;

void update(const GLint uniformId, Camera* const camera, Material* const material, Matrix4* const transform)
{
	const Matrix4& view(camera->getViewMatrix());
	const Matrix4& projection(camera->getProjectionMatrix());

	const Matrix4 modelViewProjection(transform->getTranspose() * view * projection);
	
	glUniformMatrix4fv(uniformId, 1U, false, modelViewProjection.data());
}

While we technically don't need the GL includes again as our base ShaderUniformUpdater class pulls them in, it's good practice to show what files we are actually using.. so we'll pull them in again.
Again, this file lives in our Example folder, hence the odd include paths.

There isn't much here that should be unfamiliar to you; we grab the view and projection matrix, then multiply them all together for the concatenated ModelViewProjection matrix.
The only odd bit - and the most important in this case - is the glUniformMatrix4fv call.

OpenGL has a bunch of uniform functions to send over matrices, vectors, and single variables of float, byte, int, bools, etc... it can also set "samplers" ( used for texturing, which we'll get to soon enough ) and attributes as we've seen before when setting up the Render Context. I suggest you read either the Open GL/ES spec sheet from Khronos, or either the Red Book or the OpenGL ES 2.0 Programming Guide for more information on what they do, and what they are. In this case, we're sending over a single 4x4 matrix of float values - hence the "Matrix4fv" notation.

Now we just update the example code, and we're done!

#include <cstdio>
#include <cstdlib>

#include "../../Graphics/GraphicsSystem.h"
#include "../../Graphics/Context/ShaderBasedContext.h"
#include "../../Events/EventSystem.h"
#include "../../Input/InputSystem.h"
#include "../../Input/Keyboard.h"
#include "../../Input/Pad.h"

#include "../../Graphics/Camera.h"
#include "../../Graphics/Mesh.h"
#include "../../Graphics/VertexBuffer.h"
#include "../../Graphics/IndexBuffer.h"
#include "../../Graphics/Material.h"
#include "../../Graphics/Shader.h"
#include "../../Maths/Matrix4.h"

#include "MVPUniformUpdater.h"

using namespace GLESGAE;

void controlCamera(Camera* const camera, Controller::KeyboardController* const keyboard);
Mesh* makeSprite(Shader* const shader);
Shader* makeSpriteShader();

int main(void)
{
	EventSystem* eventSystem(new EventSystem);
	InputSystem* inputSystem(new InputSystem(eventSystem));
	GraphicsSystem* graphicsSystem(new GraphicsSystem(GraphicsSystem::SHADER_BASED_RENDERING));

	if (false == graphicsSystem->initialise("GLESGAE Shader Based Test", 800, 480, 16, false)) {
		//TODO: OH NOES! WE'VE DIEDED!
		return -1;
	}
	
	graphicsSystem->getShaderContext()->addUniformUpdater("u_mvp", new MVPUniformUpdater);

	Mesh* mesh(makeSprite(makeSpriteShader()));
	Camera* camera(new Camera(Camera::CAMERA_2D));
	camera->getTransformMatrix().setPosition(Vector3(0.0F, 0.0F, -5.0F));

	eventSystem->bindToWindow(graphicsSystem->getWindow());

	Controller::KeyboardController* myKeyboard(inputSystem->newKeyboard());

	while(false == myKeyboard->getKey(Controller::KEY_ESCAPE)) {
		controlCamera(camera, myKeyboard);
		
		eventSystem->update();
		inputSystem->update();
		graphicsSystem->beginFrame();
		graphicsSystem->setCamera(camera);
		graphicsSystem->drawMesh(mesh);
		graphicsSystem->endFrame();
	}

	delete graphicsSystem;
	delete inputSystem;
	delete eventSystem;
	delete mesh;

	return 0;
}

Mesh* makeSprite(Shader* const shader)
{
	float vertexData[32] = {// Position - 16 floats
					-1.0F, 1.0F, 0.0F, 1.0F,
					1.0F, 1.0F, 0.0F, 1.0F,
					1.0F, -1.0F, 0.0F, 1.0F,
					-1.0F, -1.0F, 0.0F, 1.0F,
					// Colour - 16 floats
					0.0F, 1.0F, 0.0F, 1.0F,
					1.0F, 0.0F, 0.0F, 1.0F,
					0.0F, 0.0F, 1.0F, 1.0F,
					1.0F, 1.0F, 1.0F, 1.0F};

	unsigned int vertexSize = 32 * sizeof(float);

	unsigned char indexData[6] = { 0, 1, 2, 2, 3, 0 };
	unsigned int indexSize = 6 * sizeof(unsigned char);

	VertexBuffer* newVertexBuffer = new VertexBuffer(reinterpret_cast<unsigned char*>(&vertexData), vertexSize);
	newVertexBuffer->addFormatIdentifier(VertexBuffer::FORMAT_POSITION_4F, 4U);
	newVertexBuffer->addFormatIdentifier(VertexBuffer::FORMAT_COLOUR_4F, 4U);
	IndexBuffer* newIndexBuffer = new IndexBuffer(reinterpret_cast<unsigned char*>(&indexData), indexSize, IndexBuffer::FORMAT_UNSIGNED_BYTE);
	Material* newMaterial = new Material;
	newMaterial->setShader(shader);
	Matrix4* newTransform = new Matrix4;

	return new Mesh(newVertexBuffer, newIndexBuffer, newMaterial, newTransform);
}

void controlCamera(Camera* const camera, Controller::KeyboardController* const keyboard)
{
	Vector3 newPosition;
	camera->getTransformMatrix().getPosition(&newPosition);
	if (true == keyboard->getKey(Controller::KEY_ARROW_DOWN))
		newPosition.z() -= 0.01F;
	
	if (true == keyboard->getKey(Controller::KEY_ARROW_UP))
		newPosition.z() += 0.01F;
	
	if (true == keyboard->getKey(Controller::KEY_ARROW_LEFT))
		newPosition.x() -= 0.01F;
	
	if (true == keyboard->getKey(Controller::KEY_ARROW_RIGHT))
		newPosition.x() += 0.01F;
	camera->getTransformMatrix().setPosition(newPosition);
	
	camera->update();
}

#if defined(GLX)
	#include "../../Graphics/GLee.h"
#endif

Shader* makeSpriteShader()
{
	std::string vShader =
		"attribute vec4 a_position;			\n"
		"attribute vec4 a_colour;			\n"
		"varying vec4 v_colour;				\n"
		"uniform mat4 u_mvp;				\n"
		"void main()					\n"
		"{						\n"
		"	gl_Position = u_mvp * a_position;	\n"
		"	v_colour = a_colour;			\n"
		"}						\n";

	std::string fShader =
		#ifdef GLES2
		"precision mediump float;			\n"
		#endif
		"varying vec4 v_colour;				\n"
		"void main()					\n"
		"{						\n"
		"	gl_FragColor.grb = v_colour.rgb;	\n"
		"}						\n";

	#ifndef GLES1
		Shader* newShader(new Shader());
		newShader->createFromSource(vShader, fShader);

		return newShader;
	#else
		return 0;
	#endif
}

Most of this you should be familiar with now.. the bits we're interested in are in the vertex shader:

	std::string vShader =
		"attribute vec4 a_position;			\n"
		"attribute vec4 a_colour;			\n"
		"varying vec4 v_colour;				\n"
		"uniform mat4 u_mvp;				\n"
		"void main()					\n"
		"{						\n"
		"	gl_Position = u_mvp * a_position;	\n"
		"	v_colour = a_colour;			\n"
		"}						\n";

We've added a new uniform - a Mat4, or 4x4 matrix - called u_mvp, and then we multiply this by the position attribute we get passed in. This effectively multiplies each vertex of the mesh but the ModelViewProjection matrix, which is pretty much what the fixed function pipeline does for you.
Of course, the other function of relevance is graphicsSystem->getShaderContext()->addUniformUpdater("u_mvp", new MVPUniformUpdater); which actually adds the uniform updater in the first place, else nothing works!

Cameras are More Fun

If you run it with the 2d camera, you get the giant quad from last time ( albeit with different colours as we swizzle the RGB values in the fragment shader; just like we did in the Shader Render Context example. ) We know why this is, so we can safely ignore it.

Our camera's "right" is still in the wrong direction. Again, we know this, and again we can ignore it.

However... set the camera to be 3D and run again.
Where's the quad?
It's behind you! ... no, seriously, it is.. press the up arrow for a bit and it'll come into view.

You've got the right to go "what the smeggity smeg is going on?!" about now.
What's happened is that our camera is now pointing down the Z axis in the opposite direction than what it was in the fixed function pipeline. How? We haven't touched the camera code! And we're doing everything the Fixed Function pipeline does... right?

Actually, no...

Matrix Bashing

OpenGL multiplies in reverse order from what you expect.
So our model * view * projection matrix, actually ends up projection * view * model.
Well actually, again, it doesn't... remember in the Fixed Function pipeline there's a bunch of stacks, and importantly, your model and view matrices all sit on the same stack. If you multiply these in reverse order from top to bottom, you DO get model * view ( or model * model * model * view, etc... ) which means our final transform is really projection * modelview. Remembering that in matrix land, A * B is not necessarily B * A, we have extended fun as if we do just go ahead and shove that into our MVP updater, we still don't get anything on screen due to transpose issues.

Therefore, our "fixed" MVP updater is actually this:

void MVPUniformUpdater::update(const GLint uniformId, const Camera* const camera, const Material* const material, const Matrix4* const transform)
{
	const Matrix4& view(camera->getViewMatrix());
	const Matrix4& projection(camera->getProjectionMatrix());

	const Matrix4 modelView((*transform).getTranspose() * view.getTranspose());
	const Matrix4 modelViewProjection(projection.getTranspose() * modelView);
	
	glUniformMatrix4fv(uniformId, 1U, false, modelViewProjection.getTranspose().getData());
}

Lots of transposing, so you're right to automatically assume that this is not ideal, and probably rather heavy to calculate.
However, we can get rid of the final transpose by changing the vertex shader to gl_Position = a_position * u_mvp; because, again, A*B doesn't equal B*A and this also holds true when multiplying against vectors.

So, what can we do, instead?
If we don't transpose all the matrices and have them in the order *we* think is right ( model * view * projection ) the camera system is in left hand mode, which is the opposite of what OpenGL expects, but our Z is right in what we think ( positive down the Z axis as things get further away. )
However, if we do, the camera system is in right hand mode, which is what OpenGL expects and you can test this, as if you do use gluPerspective and glFrustum in place of my camera code, you still end up with the same result as having to transpose all these matrices.

So.. to ensure that our Z points the way we want in both Fixed Function and Shader Based contexts, we need to go fix the Fixed Function Context:

void FixedFunctionContext::setFixedFunctionCamera(Camera* const camera)
{
	glMatrixMode(GL_PROJECTION);
	glLoadMatrixf(camera->getProjectionMatrix().getData());
		
	glMatrixMode(GL_MODELVIEW);
	Matrix4& viewMatrix(camera->getViewMatrix());
	viewMatrix(0U, 2U) = -viewMatrix(0U, 2U);
	viewMatrix(1U, 2U) = -viewMatrix(1U, 2U);
	viewMatrix(2U, 2U) = -viewMatrix(2U, 2U);
	viewMatrix(3U, 2U) = -viewMatrix(3U, 2U);
	glLoadMatrixf(viewMatrix.getData());
		
	mCamera = camera;
}

And now our Fixed Function and Shader Pipelines match up!
Whew! Fun times, eh?

If you have issues understanding what's going on here, I suggest you check the code out and play about.
The engine is still nice and simple, so you should be easily able to mess about, break things, and fix things again to see how they work.

Building the Example

In the SVN there are Makefiles already setup for you.. just trigger make -f MakefileES2.pandora or whatever your chosen configuration is, and it'll happily build for you and spit out a GLESGAE.pandora binary for you to run.

Next Time

We're going to make a start on loading up textures and displaying them on our lovely test quad, then make a start at turning our current vertex array format into a vertex buffer object format.