Category Archives: Site news

Map header’s last unknown byte now known!

Earlier all documentation has stated one unknown byte in a map file header that is between player HQ locations and leaders. I now know the meaning of this byte: Map Editor sets it to 1 if the map is invalid, in it’s terms meaning there are no players set. The game reads this byte when selecting a map in Unlimited Play and ranks the map as Invalid Map. It however also shows map title and size.

I noticed this just a moment ago after having a look those demo maps that I made. I used Map Editor to generate a map of correct size, title and author information and then manually pasted the bytes with a hex editor. Only now I paid attention to what header content the maps had when I just happened to open them in a hex editor.

Because this byte is used like this I throw a recommendation that any map that works exclusively for RttR should set this byte to 0x02. This prevents users of the original game from trying to play the map file. Any non-zero value in this byte prevents the map from loading in Unlimited Play.

To validate or not to validate

I’m in the process of writing map uploading code. You might wonder why this is a slow process, but I have a very good reason: I write a lot of validation code. Why? Because I want to make absolutely sure nobody even wants to try to abuse the Map Database service. This means making the life of a spammer as hard as possible and to make things so solid that it isn’t easy to break through by code injection of any kind.

The downside is that coding is very slow. I’m also not the fastest programmer out there, but I think I write pretty solid code. And I’m not even employed as a developer, far from it!

There is also this thing called duplicate uploads. I don’t want those. So I’m writing my code in a way that there will be no two maps of the same kind, but I also have to make sure I don’t use too much processing power to do so. PHP isn’t the fastest language to work with so looping through all map data is out of the question. Just simply can’t do that. I think.

However I’m also planning on allowing revisions so that it is possible to provide a slightly changed version of an existing map. In this case both maps will have the same Map ID. This is a special ID in that it is DOS83 compatible, giving easy way to have a unique DOS filename for all the maps. I’m not forcing it though, merely it is an option when downloading a map.

User cleanup

I went through the users list and removed any account that seemed like a spam account. I hope I didn’t remove any accounts of a real user. If that happened I apologize, and you’re free to create a new account.

On the other news I found more information on how to implement a binary upload and download in AJAX, but there may be some nasty browser limitations involved. I’ll look into it in a better time as I’m having quite an uncomfortable day today as it is painful to eat or drink anything and my head is sore. Yeah, some sort of flu disease that slowly caused an aphthous ulcer (or canker sore) in a very very bad place in my mouth. Luckily the flu itself hasn’t gotten much done, only had some throat issues early last week, and again some this week. I’m not sick often so diseases tend to be quite mild on me.

Online Editor now good for map uploads!

As the Map Database part of Online Editor isn’t going to be finished very soon I made it very easy to upload maps: just upload a map and it’ll appear for everyone. At the moment any one person can have up to 15 maps in the service.


Just hit the Maps link at the bottom right corner of the top banner to find it. Decided it is clearer than “online editor” which really isn’t much of an editor at the moment.

A lot has happened today

Is it enough just to say that a lot has happened? I guess not. There is a plenty of updates on existing documentation pages and today has been a very chatty day in the comments (as you can see). We also have a new page: landscape objects table. Yes, it has all the decorative objects supported with graphical image. The table isn’t complete in the sense that it doesn’t go up to 255 that is the maximum object ID, but there really isn’t anything interesting up there. Just a lot of values that’ll cause you crashes if you’re trying to deal with mission object graphics. Regular objects are just empty after object ID 0x2B (43).

Gathering objects information

I started a new page purely about objects. The map file page would have started to become far too long if all the object details would’ve been added in so I thought it was better to start a fresh new page. Thanks to Spike for giving a little bit of new information as a push to get me started on investigating more!


Oh, see all the glitchy trees animating… without animation, but you get the idea. Each row has one tree, and as it is “animated” things go left. The leftmost comes again to become the rightmost.

BlueByte official site down

It looks like is currently offline for maintenance. This has been going on for atleast a week by now. Unfortunatenaly this also means the old original Settlers & Siedler sites are down, so the links at the bottom of this page to the official sites do not work.

However the FTP is online so you can still get your old updates and demos from an official source.

Update! works… kinda. It redirects to Ubisoft’s BlueByte Facebook page. This is a sad trend to see as people are forced to Facebook.

The problems of data and performance

In my current effort of understanding the performance issues with Online Editor’s PNG implementation I’ve noticed that there aren’t many good solutions available as far as it goes to transferring a single big chunk of data that is an entire file. Using PNG image as a data container doesn’t work as well as I wanted to thanks to PHP side being too slow in rearranging data back to the order it is supposed to be. Also some of the data lengths become quite long when sending base64 encoded data via JavaScript, even if it a compressed PNG image. This approach just has too many elements that fail on me right now, especially with big maps that I personally tend to like the most.

So this put me back into research mode and I noticed Mozilla’s FileReader documentation and it would simplify things a great deal: I no longer would have to have a separate “upload” phase to allow map editing to be possible. Instead I could just let users select a few maps, have an instant display and analysis and only after that go for any upload requirement. Also thanks to Mozilla’s pages I now know that browser support is good enough to go with these new “HTML5” technologies. The biggest issue going with this is that I need to rethink a lot of things again. Like, the component that I initially used for file upload will be of no use. Instead of sending a big chunk of data I should send multiple smaller chunks, validate them on server side and then construct the final file. Even this way some Return to the Roots 1024 x 1024 map files may cause problems and I need to implement some form of compression.

The question at this point is: is it worth the time to redesign again?

Bug fixes and added features

Online Editor has been updated. Updates since yesterday:

  • Fixed: mining resources disappeared “randomly” due to incorrect Y-location in the resource validation routine.
  • New: height randomizer has been updated to provide much richer options (gentle, medium, strong and dangerous).
  • Performance of some of the routines has been improved at the cost of more memory consumption (max. 4 MB more).
  • Editor’s user interface is now prettier.
  • Debug feature: you can now see the raw map data as an image.

Issues found since Sunday:

  • Downloading a huge map doesn’t work. This is a server side / PHP issue, the PHP code needs to be improved for memory efficiency.
  • Downloaded file doesn’t no longer gets decompressed by browser automagically. The file remains GZIP compressed after file transfer. This shouldn’t happen.
  • Setting player’s leader in the editor doesn’t change the leader (note: this change would be only visible if the map is loaded as a World Campaign mission).

Early version of Online Editor

You can now upload, change and download maps at Online Editor. It is still very far from complete, but enough to do some minor tweaks easily, like changing the world type from Greenland to Winter World. It also does some hidden magic behind the scenes, fixing a few things that can cause crashes or buggy behavior in The Settlers II.

Comments are welcome as are suggestions for highest priority features. Note that making an editor that shows the scenery like in game will take a lot of time to code so for the moment I’m pushing that idea into the distant future.