Introduction

Welcome to the "Gotcha" section of TGEA's 1.8 porting guide. If you are unfamiliar with the term "Gotcha" (quotes and all), imagine writing valid code in the engine that results in a bug or crash. The code worked in a previous version of TGEA, but is busted in 1.8. What you are experiencing is an engine modification you are not aware of messing up your work.

This is the engine's way of saying "Gotcha!" We've taken some time to document common issues you will run into while porting your previous work into TGEA 1.8, or writing your own code that is dated when compared to new GFX2 code. The most critical issues are listed in this article, but more may be added at a later date. If you run into a graphical issue that does not result in a crash or error, be sure to check this page again to see if you've fallen into a "Gotcha" situation.

Initializing GFX In Script

If you are porting an older project, or creating your own from scratch, this "Gotcha" is the first issue to address. Unlike previous versions of TGEA, in 1.8 you need to initialize the renderer from TorqueScript. This is a client side operation, so you will want to call this during your client initialization script/function:

// Called from client script
initRenderInstManager();

initRenderInstManager() is a ConsoleFunction:

ConsoleFunction(initRenderInstManager, void, 0, 0, "Initializes the render bin system and lighting system,"
               "basically a signal from script that materials/shaders are loaded and GFX is up and running.")
{
   gClientSceneGraph->initRender();
}

Before anything gets rendered to your screen, you must call the function. As an example, open GameExamples/T3D/game/scriptsAndAssets/client/init.cs. The function is called in initClient(), after the client scripts are executed but before the main menu is loaded.

Main Loop Change

This "Gotcha" is one of the first modifications you need to make if you are merging an older project. Each game project has a main.cpp file. This source file is local to your project, not the engine's source. For example, the T3D main.cpp can be found in the GameExamples/T3D/source/ directory. Within this file is the entry point for your game:

// Entry point for your game.
//
// This is build by default using the "StandardMainLoop" toolkit. Feel free
// to bring code over directly as you need to modify or extend things. You 
// will need to merge against future changes to the SML code if you do this.
S32 TorqueMain(S32 argc, const char **argv)
{
    ...
}

The actual main loop used to be a standard function call:

// Main loop
   StandardMainLoop::doMainLoop();

However, if you try running this on a 1.8 build, the application will run once and exit immediately. To rectify this, you must use a standard C while loop to keep your application running:

// Main loop
   while(StandardMainLoop::doMainLoop());

Setting Shader Constant Buffers

Before you perform a draw call, you must remember to set your shader constant buffers (GFXShaderConstBufferRef). If you forget to do this, you can end up with odd graphical glitches that do not actually produce errors/crashes/warnings.

Pseudo-Code

// Get projection matrix first
proj = GFX->getProjectionMatrix();

// Here, we are asssigning our shader constant buffer properties
myShaderConstBuffer->set(mModelViewProjSC, &proj);
myShaderConstBuffer->set(mOpacityMapSC, (S32)0);
mShaderConsts->set(mLightMapSC, (S32)1);

// *** SET SHADER CONSTANT BUFFER ***
GFX->setShaderConstBuffer(mShaderConsts);

// Now you can safely use draw calls
...

Actual Code: Precipitation

In Precipication.h, you will find various shader constant buffer variables. Here are a few examples:

GFXShaderConstBufferRef mSplashShaderConsts;
   
GFXShaderConstHandle* mSplashShaderModelViewSC;
GFXShaderConstHandle* mSplashShaderFadeStartEndSC;

The Render Delegate for Precipication is called renderObject(...). Within this function, you can see how we have modified the file to avoid this "Gotcha". First, we actually initialize the shader constant buffer:

      if (mSplashShaderConsts.isNull())
      {
         mSplashShaderConsts = mSplashShader->mShader->allocConstBuffer();

         if (mSplashShaderConsts)
         {
            mSplashShaderModelViewSC = mSplashShaderConsts->getShader()->getShaderConstHandle("$modelView");
            mSplashShaderFadeStartEndSC = mSplashShaderConsts->getShader()->getShaderConstHandle("$fadeStartEnd");
            mSplashShaderCameraPosSC = mSplashShaderConsts->getShader()->getShaderConstHandle("$cameraPos");
            mSplashShaderAmbientSC = mSplashShaderConsts->getShader()->getShaderConstHandle("$ambient");
         }
      }

In the above code, we check to see if our shader constant buffer is nullified. If it is, we allocate a buffer. If that succeeded, we initialize shader constant handles. Once everything is initialized, we address the "Gotcha" in two spots. One is right below the initialization:

      if (mSplashShaderModelViewSC)
         mSplashShaderConsts->set(mSplashShaderModelViewSC, proj);

      if (mSplashShaderFadeStartEndSC)
         mSplashShaderConsts->set(mSplashShaderFadeStartEndSC, fadeStartEnd);

The other happens further down, just before rendering occurs:

   if (mSplashShader)
   {
      if (mSplashShaderConsts.isValid())
         GFX->setShaderConstBuffer(mSplashShaderConsts);

      mSplashShader->getShader()->process();
   }

As you can see, the main point of this section is addressed. If we are using the Splash Shader, we check to see if we have a valid shader constant buffer for splash rendering. If so, GFX sets the shader constant buffer before we render.

Transpose Gotcha

Do not transpose transform matrices before passing a shader. Doing so will result in odd visuals, since you will be passing in the wrong matrix. You might see objects flickering and shooting off into infinity from your perspective. We've avoided doing this in the newer GFX2 code, but if you need to see an example of where this "Gotcha" was corrected, you can look at the fxFoliageReplicator class. Open fxFoliageReplicator.cpp.

The class's Render Delegate is renderObject(...). If you jump down to where the shader constant handles are set in this function (around line 1632), you will see the "Gotcha" being handled:

 // Set up our shader constants	
// Projection matrix
MatrixF proj = GFX->getProjectionMatrix();
//proj.transpose();

if (mFoliageShaderProjectionSC)
               mFoliageShaderConsts->set(mFoliageShaderProjectionSC, proj);

In the above code, we get the projection matrix from GFX. A few lines down, we set the shader. As you can see, we commented the transpose function. Keep this in mind when you are writing similar rendering code.

Name Consistency

This can be a very frustrating "Gotcha", as the likelihood of repeating the mistake is high. You must avoid inconsistent naming of shader variables. As an example, we are going to examine a water shader which has been updated to work with TGEA 1.8.

You will want to open GameExamples/Stronghold/game/shaders/water/waterCubeReflectRefractV.hlsl with a text editor of your choice. Pay close attention to what variables are assigned to which registers. The previous version of this shader used a different name for its light vector:

uniform float3   lightVec        : register(C21),

In past versions, you would directly set the register and what you named lightVec did not matter. You were mainly concerned with using the appropriate variables defined in shaderConsts.h. Now, using "lightVec" as the name will result in odd water rendering, such as completely black. Compare this to TGEA 1.8's version of this shader:

uniform float3   inLightVec      : register(C21),

lightVec has been renamed to inLightVec. Using the new variable name solves the bad water rendering. This "Gotcha" is easy to fix, so long as you remember the following information. If you wish to use TGEA's material system with a shader, the variables you assign to a register must match the ones used in the shader gen related source files. If you have not already tracked these files down, let's take a look.

Navigate to the engine/source/shaderGen directory. Open shaderGenVars.h and shaderGenVars.cpp. In the header, a struct called ShaderGenVars is defined. Pre-TGEA 1.8 used VC_LIGHT_DIR1, so scroll down until you see something similar. You should come across this:

const static String light1Direction;

How does this relate to the naming consistency issue? Switch to shaderGenVars.cpp and browse for light1Direction again.

const String ShaderGenVars::light1Direction("$inLightVec");

Jackpot! This is where inLightVec is assigned, and this is how the shaders tie in with TGEA's custom material system. So, to reiterate, make sure the shader variable names you use in a shader file match the ones declared in the engine. Don't forget the $ symbol, either!

Shader Compile Failure

This is a simple "Gotcha" to explain, but you need to keep an eye out for it. If your shader is not being displayed, or has bad results, check for a shader compile failure listed in the console. The engine may continue to run with only a warning about the compile failure. If this happens, go to the shader and check for any syntax errors or improper logic.

Set StateBlock

The introduction of StateBlocks introduces a new "Gotcha", but only if you are coding in the engine. When you declare a GFXStateBlocKData in script, the StateBlock is automatically set. This isn't the case when you are working in the engine. If you need an example, let's take a look at the Sky class. Open Sky.cpp, then scroll down to renderSkyBox(...).

GFX->setStateBlock(mSkyBoxSB);

The very first operation after the variables are declared is setting the StateBlock. This must happen before any rendering calls are made. Another issue with setting a StateBlock is the overriding that can occur. If you have declared a GFXStateBlock in script, it will override any C++ counterpart. For instance, if you are making changes to the terrain StateBlocks in C++, you will most likely not see any results since the TorqueScript version has taken over.

RenderDelegate

When reading about a Render Delegate, this "Gotcha" is stated but not in a way you might have noticed. Prior to TGEA 1.8, objects were automatically rendered using the same function signal. This function was renderObject(), which every object had to use if it wanted to be drawn on screen.

With the introduction of Render Delegate, this is no longer the case. This is actually a huge benefit, as you can now name an object's render function more appropriately. For instance, if you could change the Sky class's rendering function name to "renderSky", so long as you modify its Render Delegate at the time.

This brings up another "Gotcha" related to RenderDelegate. If you keep the function name "renderObject", you must change the function parameters and how it is set up. Again, let's use Sky as an example. Before 1.8, the prepRenderImage(...) class method contained this codeblock:

if (state->isObjectRendered(this)) 
{
   RenderInst *ri = gRenderInstManager.allocInst();
   ri->obj = this;
   ri->state = state;
   ri->type = RenderInstManager::RIT_Sky;
   gRenderInstManager.addInst( ri );

}

Compare that to how TGEA 1.8 sets up the RenderInst pointer:

if (state->isObjectRendered(this)) 
{
   ObjectRenderInst *ri = gRenderInstManager->allocInst<ObjectRenderInst>();
   ri->mRenderDelegate = mRenderDelegate;
   ri->state = state;
   ri->type = RenderPassManager::RIT_Sky;
   gRenderInstManager->addInst( ri );
}

There are several changes to this code block, but we will address all of them in the Sky Class Comparison document. For now, let's focus on the assignment of mRenderDelegate. Sky's RenderDelegate is assigned to Sky::renderObject(...). This function existed before TGEA 1.8 and continues to exist in the current version of TGEA. However, there is not a 1 to 1 compatibility between the versions. The parameters that are passed have changed:

Previous TGEA Version

void renderObject(SceneState *state, RenderInst *ri);

TGEA 1.8 Version

void renderObject(ObjectRenderInst *ri, BaseMatInstance* );

Instead of taking in SceneState and RenderInst pointers, the new renderObject function takes in an [[1]] and a BaseMatInstance*. The main point is previous class implementations of renderObject(...) are no longer compatible with TGEA 1.8 renderObject(...).

Remember this point or you could end up with a compiler error that makes no sense and is hard to track down. For example, Sky's RenderDelegate member variable is bound in the constructor:

mRenderDelegate.bind(this, &Sky::renderObject);

If you do not modify the old Sky::renderObject(...) parameters, you will receive this error:

error C2660: 'fastdelegate::FastDelegate2<Param1,Param2,RetType>::bind' : function does not take 2 arguments

Notice the lack of information pointing you to the renderObject(...) function.

GLSL Texture Order

By supporting OpenGL, we have to adhere to certain rules we were not obligated to follow in past versions of TGEA. One such rule is enforced when creating ShaderData objects in TorqueScript. ShaderData must now contain DX and OGL shader file references, as well as a new element.

If you open C:/torque/TGEA_1_8_Repo/GameExamples/Stronghold/game/common/clientScripts/shaders.cs, you will see a perfect example of this "Gotcha":

new ShaderData( _DebugInterior_ )
{
   DXVertexShaderFile   = "shaders/debugInteriorsV.hlsl";
   DXPixelShaderFile    = "shaders/debugInteriorsP.hlsl";
   
   OGLVertexShaderFile   = "shaders/gl/debugInteriorsV.glsl";
   OGLPixelShaderFile    = "shaders/gl/debugInteriorsP.glsl";
   
   OGLSamplerNames[0] = "$diffuseMap";
   
   pixVersion = 1.1;
};

Notice how vertex and pixel shader files are represented for both DX and OGL. That's not all. Look at the OGLSamplerNames[0] variable and the $diffuseMap value assigned to it. This is a new section of ShaderData, which is necessary because a GL shader does not know anything about texture order. Unlike a hlsl shader, it does not assume anything.

Let's compare the pixel shaders listed in the above ShaderData object. Specifically, let's focus on the diffuseMap usage:

DX (debugInteriorsP.hlsl)

//-----------------------------------------------------------------------------
// Structures                                                                  
//-----------------------------------------------------------------------------
struct ConnectData
{
   float2 texCoord        : TEXCOORD0;
};


struct Fragout
{
   float4 col : COLOR0;
};


//-----------------------------------------------------------------------------
// Main                                                                        
//-----------------------------------------------------------------------------
Fragout main( ConnectData IN,
              uniform sampler2D diffuseMap      : register(S0),
              uniform float4    shadeColor      : register(C0)
              
)
{
   Fragout OUT;

   OUT.col = shadeColor;
   OUT.col *= tex2D(diffuseMap, IN.texCoord);

   return OUT;
}

OGL (debugInteriorsP.glsl)

uniform sampler2D diffuseMap;
uniform vec4 shadeColor;

varying vec2 TEX0;

void main()
{
   vec4 diffuseColor = texture2D( diffuseMap, TEX0 );
   
   gl_FragColor = diffuseColor;
   gl_FragColor *= shadeColor;
}

As you can see in the OGL shader, the texture order is explicitly expressed in the first line of the main() function. Since you must have a OpenGL shader equivalent to the DX version, keep this texture order "Gotcha" in mind.

Shared Shader Global Space

This is a simple "Gotcha" to remember. Pixel and vertex shaders constants share the same global space. You must name your variables differently. Let's say you are using $modelView for a pixel shader const and vertex shader const. The system will read in and use the vertex shader first.

Now, if a variable contains the same data for both shader consts, you should not have a problem. However, this is not always going to be the case and you should keep this in mind.

Resource Loading Change

The new Resource Manager introduces a "Gotcha", which directly affects the loading of texture files, such as bitmaps and DDS. Previously, the GBitmap and DDSFile classes would return a class pointer. For example:

GBitmap* bitmap = GBitmap::load( path );

This is no longer the case. The new Resource Manger actually returns a resource handle. The GFXTextureManager class has a perfect example. An extremely important class function GFXTextureManager::createTexture(...). Within this function, we can see the Resource Manager in action:

// MP: Load a DDS file
Resource<DDSFile> dds = DDSFile::load( path );

// MP: Load a bitmap file
Resource<GBitmap> bitmap = GBitmap::load( path );

As you can see, instead of returning a class pointer (GBitmap* or DDSFile*), the ::load function now returns a handle to the resource, via a templated class that acts as a wrapper (Resource<class T>).

What is a CSF?

Simply put, CSF stands for Compiled Shader File. CSF files are automatically created, depending on a few conditions checked by the engine internally. CSF files have a similar functionality as DSO files.

When you compile a TorqueScript file to DSO, the game still reads it properly, but the script itself is converted to binary form making it more secure. When your shader file is compiled to CSF, it can no longer be modified by an end user.

When a shader declaration is read, the engine initializes it internally:

bool GFXD3D9Shader::init(  const Torque::Path &vertFile, 
                           const Torque::Path &pixFile, 
                           F32 pixVersion,
                           const Vector<GFXShaderMacro> &macros )
{
        ...
}

Inside the the init function, there is a check to see if the system needs to compile the shader:

if ((GFXD3DX.isLoaded) && (!Con::getBoolVariable("$shaders::forceLoadCSF", false)))
   {
      _compileShader( vertFile, vertTarget, d3dXMacros, mVertexConstBufferLayoutF, mVertexConstBufferLayoutI);
...

In the engine's source, there is a preprocessor definition named TORQUE_ENABLE_CSF_GENERATION. If this has been defined, the shader file will be compiled and saved out as a CSF:

#ifdef TORQUE_ENABLE_CSF_GENERATION
      // Ok, we've got a valid shader and constants, let's write them all out.
      if (!_saveCompiledOutput(filePath, code, bufferLayoutF, bufferLayoutI))
         Con::errorf("Unable to save shader compile output for: %s", filePath);
#endif

Material Initialization

The way materials are initialized has changed. We can look at the MatInstance class (engine/source/materials/matInstance.h) to see this modification:

Old Initialization Function

void init( SceneGraphData &dat, GFXVertexFlags vertFlags );

New Initialization Function

virtual void init(const GFXMaterialFeatureData::FeatureListHandle instanceData, const GFXMaterialFeatureData::FeatureListHandle globalData, GFXVertexFlags vertFlags);

Previously, the init(...) function took in a SceneGraphData reference which was parsed for information and features. In TGEA 1.8, we pass in two very different parameters in addition to the original vertFlags parameter:

• GFXMaterialFeatureData::FeatureListHandle instanceData - Should contain all of the features that this instance can support. For example, if it supports lightmaps then LightMap should be turned on, but if it's an instance for a mesh without lightmaps, then lightmap should be off.
• GFXMaterialFeatureData::FeatureListHandle globalData - Used to turn features on and off at a more broad level. By default, just get the one from the MaterialManager. This and instance data are AND'ed together and used with the material setup to figure out which features to turn on and off.
• GFXVertexFlags vertFlags - The vertex type to be used with this instance.

OGLSamplerNames Field Name Change

Prior to TGEA 1.8, the exposed form of the ShaderData class (ConsoleObject used in TorqueScript) used a field named "OGLSamplerNames". The internal variable used by this field was ShaderData::mOGLSamplerNames, found in engine/source/materials/shaderData.h.

void ShaderData::initPersistFields()
{
   // ... previous and remaining code not shown
    
   String                  mOGLSamplerNames[TEXTURE_STAGE_COUNT];

}

This variable was used in many places throughout the engine, but was exposed to TorqueScript in ShaderData:initPersistFields() (found in engine/source/shaderData.cpp):

addField("OGLSamplerNames",      TypeRealString,      Offset(mOGLSamplerNames, ShaderData), TEXTURE_STAGE_COUNT);

The latest version of TGEA has renamed both the class member variable and the script implementation. The following code represents the name change:

Declaration of class variable in engine/source/shaderData.h:

String                  mSamplerNames[TEXTURE_STAGE_COUNT];

Exposing variable to script in engine/source/shaderData.cpp:

void ShaderData::initPersistFields()
{
   // ... previous and remaining code not shown

   String                  mSamplerNames[TEXTURE_STAGE_COUNT];

}

Automated Mapping of samplerNames

A new class method has been added to ShaderData, which can be found in enginer/source/materials/shaderData.h/.cpp:

void mapSamplerNames(GFXShaderConstBufferRef constBuffer);

The new function definition can be found below:

//--------------------------------------------------------------------------
// mapSamplerNames
//--------------------------------------------------------------------------
void ShaderData::mapSamplerNames(GFXShaderConstBufferRef constBuffer)
{
   if (constBuffer.isNull())
      return;

   // Map the stage numbers to the constants
   for (U32 i = 0; i < TEXTURE_STAGE_COUNT; i++)
   {
      if (mTexHandlesSC[i])
         constBuffer->set(mTexHandlesSC[i], (S32)i);
   }
}

The purpose of this new method is to automate the mapping of the "samplerNames" fields from script to the actual shader buffer constants. You should use this function anytime after you use GFXShader::allocConstBuffer() and pass it in your newly created GFXShaderConstBufferRef so it can map the actual texture stage values into your constant buffer's parameters.

You will need to do this to allow GLSL shaders to properly sample from the right texture register. A simple example can be found in GroundCover class, specifically GroundCover::_initShader(). Open up engine/source/T3D/fx/groundCover.cpp. Scroll down to the function I just mentioned. Halfway through the function, after the samplers are set up, you will see the example:

void GroundCover::_initShader()
{
   // ... previous and remaining code not shown

   mConstBuffer = mBBShader->mShader->allocConstBuffer();

   if (mConstBuffer)
      mBBShader->mapSamplerNames(mConstBuffer);

}

As you can see, once the allocConstBuffer is called successfully we call mapSamplerNames with thew newly defined mConstBuffer.

November 2008 DirectX SDK Shader Bug

There was a bug in the shader compiler in the November 2008 DirectX SDK (as referenced in this GarageGames forum thread: TGEA 1.7.1 and DirectX Nov. 2008 - broken ). A workaround was added for it that is specific to the Nov 2008 DXSDK (also detailed in that thread).

However, we recommend you stick with the August 2008 DXSDK (also linked in that thread). Our contacts at Microsoft informed us the bug report was recorded and should be fixed in the March 2009 DirectX SDK.

Atlas module not included in project template

The Atlas module is by default not included in the project template and thus in projects set up by SetupNewProject.exe. This is intentional as Atlas is not supported on all platforms at the moment.

To include the module in your project, edit the file config/project.conf in your project folder and uncomment the line that reads

	//includeModule( 'atlas' );

After this, regenerate your project files by invoking generateProjects.bat (Windows) or generateProjects.command (Mac). Don't forget to build your project afterwards in the IDE you are using.