Category Archives: Tools

Getting into something usable

My weekend has a clear goal: by the end of Sunday I’ve setup a small online map editor service that allows anyone to upload a WLD/SWD map to their browser for editing, do changes and save the map again to their computer. Once this is possible I can keep on adding more features to the editor over time.

Map handling updates

Now that the technical side is no longer such a big of a problem I have rebuilt the JavaScript code in a more object oriented manner to make it easier to manipulate map data. I added a few features to showcase things that are possible to do:

  • You can now change terrain type in real time.
  • You can switch between two different palettes.
  • You can manipulate a map’s height map with some randomness – and see the changes happen.
  • You can try a few maps instead of seeing just one.

These aren’t very major yet, but it gives a little bit of something. Some important stuff is to be done, like recalculating the part of map data that deals with buildable areas and landmasses. Knowing this information and calculating it correctly is very important for some future features, such as placement of headquarters or finding out about possible harbor sites. Deeper validation of maps also benefits of knowing more. There are also some bugs in the original game that can be fixed by disallowing stuff that causes these bugs to appear, such as placement of granite over water/lava/snow/swamp or Map Editor’s bug of placing fish to unwalkable parts of coastline.

Visit the Canvas Map Viewer to see the current progress.

Canvas limitations conquered

So, today I noticed a serious issue with my earlier PNG implementation: HTML5 Canvas doesn’t treat colors as exactly as I’d like to when alpha channel is active on a pixel. This results RGB data on a semitransparent pixel to be incorrect when reading it. So I can’t use a PNG image with alpha transparency.

Now I’ve sorted out this issue by making the image larger, dropping all 7-bit stuff and providing everything as a 24-bit PNG image. This actually compresses better than image with an alpha channel which is nice.

Besides fixing the PHP side of things I have of course been working with the JavaScript side of things. I now read the PNG file without issues, although gotta remember canvas always provides alpha channel information that just has to be ignored.

You can find the PHP source as well as an example page at http://settlers2.net/canvas/

Setting a PNG standard

I have successfully completed a WLD/SWD/DAT to PNG world converter! A resulting file:

Europe World Campaign

It doesn’t look like much, but it contains all the information one needs to restore it back into a complete WLD/SWD file! It just needs a lot of recalculations for some data that has been left out.

Left side block:

  • R = height map
  • G = texture 1
  • B = texture 2

Right side block:

  • R = object 1
  • G = object 2
  • B = resources

Alpha channel (in PHP’s limited 7-bit mode) runs animal information 5 bytes per animal:

  • 1 byte ID
  • Low 7-bits of X
  • Next 7-bits of X (= 14 bits, loss of 2 highest bits)
  • Low 7-bits of Y
  • Next 7-bits of Y (= 14 bits, loss of 2 highest bits)

However, last 298 bytes are reserved for extra map information. Of this only 84 bytes is actually used for data, the remaining 214 bytes are left for possible future expansion. This means the smallest 32 x 32 map can have up to 350 animals. That should be enough!

The data is formatted as such:

  • Title, 23 bytes (lowest 7-bits of each character)
  • Author, 19 bytes (lowest 7-bits of each character)
  • = Total of 42 bytes
  • Carry over bits, 6 bytes (highest 7-bits, one bit per character)
  • = Total of 42 bits
  • Map type, 1 byte (greenland = 0, wasteland = 1, winter = 2)
  • Player face, 7 bytes (0 to 13)
  • Player HQ X pos, 14 bytes (halved to two 7-bit bytes like animal X)
  • Player HQ Y pos, 14 bytes (halved to two 7-bit bytes like animal Y)

And that is it. Everything else in a Settlers II map file is something that can be recalculated based on this information.

Download removed, outdated because of HTML5 canvas alpha channel issues

 

And then I need to start experimenting with HTML5 Canvas & JavaScript…

The missing pieces of map file format

After finding out about RttR documentation and having a look at some of the source code we now have gathered a great deal of information about The Settlers II WLD/SWD file format. Yet there is still things that need further investigation:

  • The single unknown byte in file header, what is it for?
    A wild guess at this point without testing: maybe it could set the MISS#BOB graphics? That is a feature that could’ve been added from RTX files just like the leaders…
  • Block 8: being always just full of NULL this probably needs a look from savegame files. We haven’t had a proper look at the savegame files yet!
  • Block 10: this is always full of 7. Since there is a maximum of seven players in the game maybe values 0 to 6 could indicate the player who owns that spot? So an ownership block. But this can only be verified by having a look at the savegame files.
  • Block 11: I can hardly believe this would be used only for Map Editor’s cursor position. So I throw a wild guess: maybe it is the highlighted icons map! This would be internal game use only, and the Map Editor simply saves memory as it is at that very moment, explaining why we appear to get the “cursor position”.

Return to the Roots project has done some job in other areas. A simple thing they’ve done with the RttR Map Editor is to allow for bigger maps, up to 1024 x 1024 (which is way bigger than the maximum of 256 x 256 the original game can handle). The problem is that this breaks backwards compatibility!

Continue reading

Plan revealed: an online map generator & mission builder

I’ve been thinking about this for some time now. While the game is old, there is still atleast some life on it thanks to GOG.com and especially thanks to the very active German community of Settlers fans. Just having a look at Return to the Roots forum gives a good view of what is happening: there is even an entirely new people being developed, and graphics quality wise they’re doing a pretty good job!

Initially I’m thinking on making the map tool only to create files that are compatible with the original Settlers II. However, this should also be very compatible with RttR, so that should be good news if my project gets anywhere. Most of the initial “mission support” comes from harbors. However, supporting missions will eventually mean providing whole campaigns, because changing missions easily means changing a whole lot of files starting from mission resource text files (RTX) to mission string files (ENG/GER) to core string files. With existing 2NDpaign campaign you even need to change some graphics files! So the tool really needs to be able to cover a lot of ground besides just being a replacement map editor that works online.

Technically I’ve already decided a few things. First of all I’ll be using the new HTML5 Canvas, which means maps are generated and edited using JavaScript code. The map data will be transmitted as PNG images for minimal transfer sizes between the browser and server. I’ll need to create a custom format/implementation for that. For the least there is no point delivering the WLD/SWD files unmodified, that is just a waste of bandwidth!

I don’t know yet how quickly the development will proceed. But I’ll eventually start working on it.

Working on a new tool

I did some work on reading Settlers II language files in PHP. I guess I’ll eventually come up with an online localization tool. That should make it very easy to translate the game as you can just first translate the game, then download a single file and extract that to the game folder -> game translated.

Nothing to show at this point though, will see if I get this done, and I don’t know yet how I manage with user permissions etc.