TSE/Atlas/Create a Terrain/Freeworld3D
From TDN
|
[edit] IntroductionFreeworld3d can be quirky, slow, crash prone, and hog memory, but it is still one of the few good options for hand crafting terrains for Atlas. With Freeworld3D you do not have to use fractal based tools like L3DT or Terragen. This gives you an experience similar to the old TGE in-engine terrain editor. With the release of Atlas2 you can also create blended terrains in Freeworld, but take note that it will require some code changes. [edit] Creating TerrainWe're not going to go into detail here on how to use Freeworld3D tools to make terrains. You can figure that one out on your own as it is rather easy to use. What we will do is provide you a few tips to make the process easier and to keep you from having issues importing it into Atlas. [edit] New ProjectWhen creating a new project in Freeworld3D your presented with the following dialog:
The Units Per Step value needs a little explanation. It is simply the number of world units between grid points on the terrain. This is something of use within Freeworld and is not necessary for Atlas terrain. I suggest keeping this value at 1. Also you should always keep Enable LOD on... it doesn't effect your final terrain, but it improves performance considerable when editing. |
|
Unique vs. Blended Texturing
More details on this topic can be found over here. The short of it is a tradeoff between control and disk space in most cases. Unique texturing gives you terrain as detailed as you like it with full control per-pixel at a high disk space cost. Blended terrain can give you high detail with less per-pixel control, but is very compact on disk. Take a look at these examples:
blah blah blah blah about the examples.
The problem with using unique texturing in Freeworld is that it will only export a maximum of a 4096x4096 texture. This means that if your source textures are 128x128 you can only have a maximum UV scale of 32 before you start loosing resolution in your output texture. I've found you can fudge this some, but don't expect to get high detail from Freeworld with this technique unless your creating a pretty small terrain. You pretty much have to use a detail texture in Atlas (broken at the time of this writing) for you to get decent results for large terrains.
Blended texturing is a better bet for exporting from Freeworld, but certain code modifications and rules need to be followed to get good results. First you are limited to one Base Layer and three Alpha Layers as at this time Atlas only supports 4 layers in a blended terrain. Also you must keep a consistent texture resolution between layers (the texture size multiplied by the UV scale should be the same for each later). Finally you must limit yourself to no larger than 1024 alpha maps for each layer (Torque will not load larger PNGs). You'll better understand why these limits are in place as you get to the export steps.
Lightmaps
As of this writing Atlas2 does not support terrain lighting in any way (neither vertex lighting or shadows). This means you must bake in your direct and ambient sun light into a lightmap. First thing you should do is disable Freeworld's vertex based lighting via the Settings tab in the right side pane. Uncheck 'Enabled' in the 'Lighting' section.
If you don't disable this you'll end up seeing a combination of the vertex lighting Freeworld does terrain plus your generated lightmap. This gives you a false sense of the final look of the lightmap. So now you can generate your lightmap via the Terrain->Lightmap Generator menu.
Keep the Compress checkbox on... it doesn't effect your output lightmap, but improves performance in Freeworld. Be sure you enable Slope Shadows as well as Terrain Shadows which gives you both baked in vertex lighting and terrain self shadowing. Finally make sure that the Modulate radio button is active. You can now Generate then Apply the lightmap.
Exporting The Terrain
The easiest way to export all the maps and data needed for importing into Atlas is via the File->Export->Custom Export menu. You'll be presented with the following dialog:
Export into scratch area as this process will generate several files and folders in the destination folder. On the File Settings tab leave ASCII enabled and on the Scene tab make sure Terrain and Terrain Layers are checked off. Finally on the Terrain / Meshes / Vegetation tab set the Heightmap Format to 16bit Raw and Terrain Layers to JPG then hit Export. In the export folder you'll get a world.cfg file and several folders.
Unique Texturing
For unique texturing you have an additional export step.
Importing Into Atlas2
Now that you have all the data you need exported from Freeworld you can start the import process. We'll present a script for automating this process in a later section, but for now lets do it step by step.
Heightmap
The first part of the terrain to build is the geometry from the heightmap data. This step is the same regardless of the texturing method you used. If you look in the export directory you'll see a folder called terrain and in it you'll find the file heightmap16bit.raw. Copy this file into your Torque YOUR_MOD_FOLDER/data/terrains folder (replace YOUR_MOD_FOLDER with whatever your mod name is). Before we can import it there are a few values we need to know.
Size And Height Scale
Two of the parameters needed when converting the heightmap into geometry is its size and height scale. In the 16bit RAW file each pixel is a value between 0 and 65536... 0 being the lowest terrain point and 65536 being the highest. The height scale converts the value in the height map into a meter scale unit. The size is simply the dimensions of the RAW file.
Freeworld provides these values for you in the world.cfg found in the export directory. Open that up with notepad and you'll find a section similar to the following:
<Terrain Size="1024" Step="1.000000"> <Heightmap File="/terrain/heightmap16bit.raw" Format="16bit RAW" Scale="0.002777"/> </Terrain>
The Size="1024" and Scale="0.002777" bits are the parts your interested in here. These values will be needed later in the command to generate the Atlas chunk file.
Meters Per Pixel
Another value you'll need for chunk file generation is how many meters of terrain to generate per heightmap pixel. Freeworld units are pretty arbitrary and it's up to you to decide on the final Torque size. Consider that a 1024 heightmap at 1 meter per pixel is 1 sq km (or a little under 2/3 of a mile). It may just take a few imports and play tests before you decide. Remember that this value can also be fractional.
Height Error
This value is used to decide how much error, error being deviation from the heightmap, to allow in the mesh generated from the heightmap. Too large an error tolerance and you will get significant visual popping of LODs and the mesh won't follow the heightmap closely. Too small and your performance is terrible and your final atlas file will be huge. In general i've found that between 1 or 2 is a good error value. Others have suggested that 1% of the terrain height is a good number. You'll have to experiment some here and see what works for your particular terrain.
Tree Depth
The tree depth for the generated terrain quadtree is another parameter needed during import. This value is chiefly used to balance the number of vertices per LOD mesh in the terrain. When you export you'll see a graph like this one:
=== Chunk Statistics ===
===== Level Count Avg. Size
0 256 761.562500
1 64 2978.140625
2 16 8176.562500
3 4 22972.250000
4 1 73725.000000
====================================
chunks: 341
input verts: 4194304
output verts: 58817
avg verts/chunk: 172
output bytes: 12645
bytes/input vert: 0.00
bytes/output vert: 0.21
real triangles: 682000
In this example we specified a tree depth of 4. You can see the Avg. Size (actually average triangles) in each terrain LOD level. Ideally you want more than 1000 vertices per level (since the triangles are rendered as strips you can estimate verts = triangle count * 2), but it isn't mandatory. Also you want to keep each level under 32K vertices. Many times this isn't possible with the top levels of LOD like in this example without increasing the error tolerance. As long as you never get far enough away from the terrain to show the top level LOD block this doesn't matter.
So basically this is a trial and error process. I suggest starting with a value of 4 to 6 and doing a few imports before you settle on one value. If you wanted to be really through you could go as far as measuring frame rate with different tree levels, but in general this isn't necessary.
Generate The Terrain
Now we have all the values we need to generate the terrain. Launch Torque and open up the console and enter the following command substituting in your values:
atlasOldGenerateChunkFileFromRaw16( "YOUR_MOD_FOLDER/data/terrains/heightmap16bit.raw",
HEIGHTMAP_SIZE,
METERS_PER_PIXEL,
HEIGHT_SCALE,
"YOUR_MOD_FOLDER/data/terrains/deleteme.geometry.chu",
HEIGHT_ERROR,
TREE_DEPTH );
A tip, don't take the "AtlasOldActivationHeightfield::generateNodeData - Max exceeded! May have paging issues!" and "NOTE: verts/chunk is low; for higher poly throughput consider setting the tree depth to X and reprocessing." messages too seriously... it's just a suggestion and not required. In fact in most cases there isn't a perfect setting which will not result in these warnings.
So we have a CHU file and need to convert it into an Atlas2 file. Enter the following into the console:
importOldAtlasCHU( "YOUR_MOD_FOLDER/data/terrains/deleteme.geometry.chu",
"YOUR_MOD_FOLDER/data/terrains/deleteme.geometry.atlas" )
Now you have an Atlas terrain geometry in a temporary .atlas file and can start importing the textures.
Unique Texture
Blended Texture
The process for importing blended textures involves creating two different .atlas files; a lightmap and the opacity map.
Lightmap
The lightmap is pretty straight forward to process. Place the Lightmap0.jpg from your Freeworld export terrain folder into the Torque terrains folder. Now launch Torque, open up the console, and enter the following command substituting in your mod folder:
atlasGenerateTextureTOCFromTiles( 1,
"YOUR_MOD_FOLDER/data/terrains/lightmap0.jpg",
"YOUR_MOD_FOLDER/data/terrains/deleteme.lightmap.atlas",
0 );
A new .atlas file will be generated containing the lightmap in an Atlas2 compatible format. This will be used later when generating the final blended terrain atlas file.
Opacity Map
If you look in your terrain Freeworld export folder you'll see one or more XXXX_alpha.raw files. These are the opacity maps which are used to define what areas to render different tile layers. Before importing into Atlas2 you need to take these individual maps and place them into a single PNG.
So fire up your favorate image manipulation tool (Photoshop or Gimp is recommended) and combine the RAW images into one image. If you used a Base Layer for the first terrain layer you need to make the red color channel pure white as the base layer is always on. The other layers need to be placed into the map from the lowest to the highest; Green, Blue, and Alpha. This is why there is currently a limit on terrain layers... storage of the opacity maps into a single texture. In the future a command may be provided to automatically build the opacity map from multiple layer images. Especially once the 4 layer limitation is lifted. For now just save this image out as a 32bit PNG (for Photoshop users you will need the SuperPNG export plugin).
So you should now have a file in your Torque terrain folder called opacity.png which contains the combined 4 opacity maps for your terrain. Now the next part requires one of the code changes we discussed earlier. The second parameter in the new atlasGenerateOpacityTOC command should be true if you plan on using the standard Atlas blender shader verses the lerp blend shader. Go ahead and launch Torque, open up the console, and enter the following command substituting in your mod folder:
atlasGenerateOpacityTOC( "YOUR_MOD_FOLDER/data/terrains/opacity.png",
true,
"YOUR_MOD_FOLDER/data/terrains/deleteme.opacity.atlas" );
And that's it for the opacity map. Before we can generate the final blender terrain we need to discuss one importaint parameter.
Virtual Map
You can think of the virtual map size as the size of the texture generated dynamically during blending. Of course this isn't a real texture limited by hardware features and can be as big as it needs to be. The virtual map size will determine the repetition of your layer textures and therefore the amount of detail in your final terrain texture. Earlier in this article i mentioned that you must keep a consistent texture resolution between layers... this is why.
So to calculate your virtual map size you need to take the texture size multiplied by the UV Scale for any one of your layers. Now we're ready for the final step in creating the blender terrain.
Generate Blend Terrain
This final step takes all the individual temporary .atlas files and creates the terrain. Again launch Torque, open up the console, and enter the following command substituting in your parameters:
atlasGenerateBlenderTerrain( "YOUR_MOD_FOLDER/data/terrains/terrain.atlas",
"YOUR_MOD_FOLDER/data/terrains/deleteme.geometry.atlas",
"YOUR_MOD_FOLDER/data/terrains/deleteme.opacity.atlas",
"YOUR_MOD_FOLDER/data/terrains/deleteme.lightmap.atlas",
VIRTUAL_MAP_SIZE,
"YOUR_MOD_FOLDER/data/terrains/LAYER_TILE1",
"YOUR_MOD_FOLDER/data/terrains/LAYER_TILE2",
"YOUR_MOD_FOLDER/data/terrains/LAYER_TILE3",
"YOUR_MOD_FOLDER/data/terrains/LAYER_TILE4" );
Once this command completes you can delete all the deleteme. files and you only need the final terrain.atlas and the individual layer tile textures. Your done with the import process and can now look at your terrain in game.
Loading Atlas2 Terrain
new AtlasInstance2(NewTerrain) {
position = "0 0 0";
rotation = "0 0 1 0";
scale = "1 1 1";
atlasFile = "~/data/terrains/terrain.atlas";
};
AtlasBuild Script
In the process of figuring this all out i created a script which performs all the steps here in the right order and has a section at the top allowing you to tweak all the input parameters. To use this script create a atlasbuild.cs in the same folder as the main.cs script and paste the following code into it. Then launch Torque with the command line: TSE.exe atlasbuild.cs. Once it runs be sure to check both the output folder for the final terrain.atlas file and read thru the console.log looking for error messages.
//-----------------------------------------------------------------------------
// Atlas Build
// Copyright (C) Sickhead Games, LLC.
//-----------------------------------------------------------------------------
// It might be helpful to read the acompanying article on TDN
// to help understand the process and the various parameters:
//
// http://tdn.garagegames.com/wiki/TSE/Atlas/Create_a_Terrain/Freeworld3D
///////////////////////////////////////////////////////////////////////////////
/// Config Begin
///////////////////////////////////////////////////////////////////////////////
/// The mod folder path to where your input terrain
/// data files are. We expect this folder to be a
/// subfolder of the terrain folder and should include
/// the following files:
///
/// heightmap.raw
///
/// If your doing unique textured terrain you must have
/// the following files:
///
/// colormap.jpg
///
/// If your doing blend texture terrain you must have
/// the following files:
///
/// lightmap.jpg
/// opacity.png
/// tile1.jpg
/// tile2.jpg
/// tile3.jpg
/// tile4.jpg
///
%terrainPath = "demo/data/terrains/TERRAIN_NAME/";
/// The pixel size of the terrain RAW minus 1. This
/// value must be power of two square!
%terrainSize = 1024;
/// The number of levels to the final tree for both
/// the terrain geometry and the texture.
%terrainLevels = 5;
/// The number of output meters per heightmap pixel.
%terrainMetersPerPixel = 1;
/// The best way to export from freeworld is via the dialog
/// under 'File->Export->Custom'. It generates all the maps
/// and a 'world.cfg' file:
///
/// <Terrain Size="1025" Step="1.000000">
/// <Heightmap File="/terrain/heightmap16bit.raw"
/// Format="16bit RAW" Scale="0.000397"/>
/// </Terrain>
///
/// The scale above goes into the terrain scale here.
%terrainScale = 0.002777;
/// The amount of error to allow when creating terrain
/// chunk LODs. A good number is 1% of the unscaled height.
%terrainError = ( 65536 * %terrainScale ) * 0.01;
/// If your using blended terrain then set this to true
/// else set it to false for unique terrain.
%blendTerrain = true;
if ( %blendTerrain )
{
/// The virtual map is the texture that the blender is
/// creating at runtime. Its size will determin the
/// repetition of your layer textures. For this reason
/// you must use the UV scale and texture sizes when
/// creating your terrain in Freeworld. Sucks... but
/// that is the way it is at the moment with Atlas2.
/// To get the virtual map size multiply your texture
/// size by your UV scale.
%virtualMapSize = 512 * 128;
}
///////////////////////////////////////////////////////////////////////////////
/// Config End
///////////////////////////////////////////////////////////////////////////////
enableWinConsole(true);
// Start the console logger in a locked
// mode and clear the junk at the top.
setLogMode(2);
cls();
echo( "Starting Atlas Build!\n\n" );
echo( "Calling atlasOldGenerateChunkFileFromRaw16!\n" );
echo( "-------------------------------------------\n\n" );
%rawPath = %terrainPath @ "heightmap.raw";
%chuPath = %terrainPath @ "deleteme.geometry.chu";
atlasOldGenerateChunkFileFromRaw16( %rawPath,
%terrainSize,
%terrainMetersPerPixel,
%terrainScale,
%chuPath,
%terrainError,
%terrainLevels );
echo( "\n\n" );
echo( "Calling importOldAtlasCHU!\n" );
echo( "-------------------------------------------\n\n" );
%geomPath = %terrainPath @ "deleteme.geometry.atlas";
importOldAtlasCHU( %chuPath,
%geomPath );
%atlasPath = %terrainPath @ "terrain.atlas";
if ( %blendTerrain )
{
echo( "\n\n" );
echo( "Calling atlasGenerateOpacityTOC!\n" );
echo( "-------------------------------------------\n\n" );
%opacityPath = %terrainPath @ "deleteme.opacity.atlas";
atlasGenerateOpacityTOC( %terrainPath @ "opacity.png",
true,
%opacityPath );
echo( "\n\n" );
echo( "Calling atlasGenerateTextureTOCFromTiles for lightmap!\n" );
echo( "-------------------------------------------\n\n" );
%lightmapPath = %terrainPath @ "deleteme.lightmap.atlas";
atlasGenerateTextureTOCFromTiles( 1,
%terrainPath @ "lightmap",
%lightmapPath,
0 );
echo( "\n\n" );
echo( "Calling atlasGenerateBlenderTerrain!\n" );
echo( "-------------------------------------------\n\n" );
atlasGenerateBlenderTerrain( %atlasPath,
%geomPath,
%opacityPath,
%lightmapPath,
%virtualMapSize,
%terrainPath @ "tile1",
%terrainPath @ "tile2",
%terrainPath @ "tile3",
%terrainPath @ "tile4" );
}
else
{
echo( "\n\n" );
echo( "Calling atlasGenerateTextureTOCFromLargeJPEG!\n" );
echo( "-------------------------------------------\n\n" );
%colorPath = %terrainPath @ "deleteme.color.atlas";
atlasGenerateTextureTOCFromLargeJPEG( %terrainPath @ "colormap.jpg",
%terrainLevels,
%colorPath );
echo( "\n\n" );
echo( "Calling atlasGenerateUniqueTerrain!\n" );
echo( "-------------------------------------------\n\n" );
atlasGenerateUniqueTerrain( %atlasPath,
%geomPath,
%colorPath );
}
echo( "\n\n" );
echo( "-------------------------------------------\n" );
echo( "Finished Atlas Build!\n\n" );
function onExit()
{
}
// We're done!
quit();
Code Changes
This section is really for the coders in the audience. As of the time of this writing Atlas2 is still very new and it does have some issues with blend texture terrains. If you plan on using blend textures there are a few code changes that need to be made to the engine.
Additive Opacity
The blender in Atlas2 uses a simple additive blending method. This means that if the total opacity value for all the terrain layers added together is greater than 1, your blended pixel will be oversaturated.
There are two ways to solve this issue and your decision is a trade off between performance, compatibility, and quality.
Lerp Blend
If your willing to sacrafice some performance and 1.1 pixel shader compatibility, a lerp'd additive blend will give you accurate per-pixel blends that very closely resemble the terrain within Freeworld. The change is simple in that only the pixel shader needs to be changed and nothing needs to happen in the engine. If you wish to go down this route open up the shaders/atlas/atlasBlenderPS20P.hlsl file and merge the following changes:
//*****************************************************************************
// Lightmap / cubemap shader
//*****************************************************************************
// Enable this if your opacity maps are not
// setup for additive blending.
#define LERP_BLEND
//-----------------------------------------------------------------------------
// Data
//-----------------------------------------------------------------------------
struct v2f
{
float4 HPOS : POSITION;
float2 tex : TEXCOORD0;
float2 lmTex : TEXCOORD1;
float2 tex1 : TEXCOORD2;
float2 tex2 : TEXCOORD3;
float2 tex3 : TEXCOORD4;
float2 tex4 : TEXCOORD5;
};
struct Fragout
{
float4 col : COLOR0;
};
//-----------------------------------------------------------------------------
// Main
//-----------------------------------------------------------------------------
Fragout main(v2f IN,
uniform sampler2D opacity : register(S0),
uniform sampler2D lightMap : register(S1),
uniform sampler2D tex1 : register(S2),
uniform sampler2D tex2 : register(S3),
uniform sampler2D tex3 : register(S4),
uniform sampler2D tex4 : register(S5)
)
{
Fragout OUT;
// Get colors
float4 o = tex2D(opacity, IN.tex);
float4 t1 = tex2D(tex1, IN.tex1);
float4 t2 = tex2D(tex2, IN.tex2);
float4 t3 = tex2D(tex3, IN.tex3);
float4 t4 = tex2D(tex4, IN.tex4);
float4 lm = tex2D(lightMap, IN.lmTex);
#ifdef LERP_BLEND
// Lerp additive blend.
float4 c = t1 * o.r;
c = lerp( c, t2, o.g );
c = lerp( c, t3, o.b );
c = lerp( c, t4, o.a );
OUT.col = lm * c;
#else
// Additive blend.
OUT.col = lm * (o.r * t1 + o.g * t2 + o.b * t3 + o.a * t4);
#endif
return OUT;
}
Technically this shader could be improved so that a console field can be added to the AtlasInstance2 that could toggle the blend method with a uniform boolean passed to the shader. This exercise is left up to the reader, but feel free to update the article with the necessary changes.
atlasGenerateOpacityTOC
To that ends i've added the following function to the bottom of engine/atlas/editor/atlasImportTiles.cpp:
ConsoleFunction(atlasGenerateOpacityTOC, bool, 4, 4, "(inFile, makeAdditive, outFile)\n"
"Generates an opacity TOC and optionally modifies it to work with non-lerp'ed additive blending.")
{
const char *inPath = argv[1];
const bool makeAdditive = dAtob( argv[2] );
const char *outPath = argv[3];
GBitmap *bitmap = GBitmap::load( inPath );
if ( !bitmap )
{
Con::errorf( "atlasGenerateOpacityTOC - unable to open file '%s'!", inPath );
return false;
}
if ( bitmap->getFormat() != GFXFormatR8G8B8A8 )
{
Con::errorf( "atlasGenerateOpacityTOC - expecting 32bit PNG!" );
return false;
}
if ( makeAdditive )
{
const U32 width = bitmap->getWidth();
const U32 height = bitmap->getHeight();
U8* bits = bitmap->getWritableBits();
for ( U32 w=0; w < width; w++ )
{
for ( U32 h=0; h < height; h++, bits += 4 )
{
if ( bits[3] == U8_MAX )
{
bits[0] = bits[1] = bits[2] = 0;
}
else if ( bits[3] + bits[2] >= U8_MAX )
{
bits[2] = U8_MAX - bits[3];
bits[0] = bits[1] = 0;
}
else if ( bits[3] + bits[2] + bits[1] >= U8_MAX )
{
bits[1] = U8_MAX - ( bits[3] + bits[2] );
bits[0] = 0;
}
else if ( bits[3] + bits[2] + bits[1] + bits[0] >= U8_MAX )
{
bits[0] = U8_MAX - ( bits[3] + bits[2] + bits[1] );
}
}
}
}
// Allocate a new AtlasFile.
AtlasFile af;
// Put a new textureTOC in it.
AtlasResourceTexTOC *arttoc = new AtlasResourceTexTOC;
const U32 treeDepth = getBinLog2(1) + 1;
arttoc->initializeTOC(treeDepth);
af.registerTOC(arttoc);
// Write TOCs out and get ready to do IO.
if(!af.createNew(outPath))
{
Con::errorf("atlasGenerateOpacityTOC - Could not create new Atlas file '%s'", outPath);
}
af.startLoaderThreads();
// Create a chunk to store this data in.
AtlasTexChunk *atc = new AtlasTexChunk;
atc->mFormat = AtlasTexChunk::FormatPNG;
atc->bitmap = bitmap;
// And store the chunk.
AtlasResourceTexStub *tStub = arttoc->getStub(0, Point2I(0,0));
arttoc->instateNewChunk(tStub, atc, true);
tStub->purge();
arttoc->generate(RectI(0, 0, 1, 1));
af.waitForPendingWrites();
return true;
}
UV Scale Fix
Additional Tips
Here are some tips for creating good Atlas terrains in Freeworld3D:
- Don't waste terrain by putting a small island in the middle of a 2048 map... it just wastes disk space and performance in game.
- Before you get into the details of your terrain do a test in game with the basic terrain layout and just the single base map. This lets you get a feel for scale and resolution of the texture before you spend time fleshing it out.
Categories: TGEA | Atlas | Freeworld3D
