RunUO Community

This is a sample guest message. Register a free account today to become a member! Once signed in, you'll be able to participate on this site by adding your own topics and posts, as well as connect with other members through your own private inbox!

Ultima Live - In Game Map Editing Framework

Praxiiz

Sorceror
The grid using client side statics would only be viewable by the person who brings it up. This is because razor sends the packets to the client directly, without any help from the server at all. The server would have no knowledge of them at all.

The screenshot that I showed was using bloodtiles hued to alternating colors. The grid would be controlled either by command or using the razor tab. At this point, I'm still trying to decide which would be more effective.
 

Pure Insanity

Sorceror
Command, that way it's server side. And any shard owner could get creative and do as they wish with it. So far I'm loving the system, really making a lot of stuff much easier. Now I can have staff do stuff that normally I was only able to ever do.
 

Talow

Sorceror
Could you just make it so you could use a command to change the tiles that are being played with. Keep a back up of what they should be, and when done playing set the tile back to the type of tile it should be? Then you shouldn't need to fuss much with any of that part but it would be the staffer's responsibility to set them back.
 

-hash-

Sorceror
Sorry, I've been a little slow...

I've only briefly worked with the version manager in the past while documenting packets. Took me a little while to actually code everything together into a working system to make sure my notes were all correct.

Basically, the general idea is:

The client/server keeps track of version numbers for the following files:

1. Artwork
2. Tiledata
3. Animdata
4. Hues
5. Regions
6. Skills
7. Multis
8. Terrain
9. Statics

Every entry's version number is logged. These numbers are grouped together in eights and hashed several times into smaller and smaller blocks. When the client logs in, it sends only four 0x4B (Check Version) packets. If any of these hashes don't match the server's cache, the server asks the client for the relevant hashes one chain up, and so on, and so on.... In the end, the server is able to identify the specific entries in the above files that are mismatched and send the appropriate update packets. So, instead of checking hundreds of thousands of version numbers, you get away with checking only a few.

The server can restart this process at any point, check any arbitrary version number, etc. The God Client even lets you walk around like normal as this process takes place.

I'm still working on a comprehensive document detailing the specifics of the system--probably won't have it finished until tomorrow.

In the meantime, here's a quickly cobbled-together video demonstrating it:
http://www.youtube.com/watch?v=RFi5d7XWDWY

Still, this is mostly an exercise in "Theoretical Ultima" as it concerns your project since the latest clients don't have this functionality :p

But, it does offer one solution to the versioning problem. It would just be up to you to handle the packet flow.
 

Praxiiz

Sorceror
The god client looks amazing in action. I wish I could patch in that kind of functionality. I really appreciate this information.
 

Obsidian Fire

Sorceror
Question? I heard that a God Client shipped with the Orginal version of Ultima Online, Would that be "Ultima Online Windows 95 client Release version. September 1997" Ultima Online Renaissance? and if so Whats the *.EXE called? or is it the regular game client but needs a Command line to start it as the God Client?
 

clark71822

Sorceror
It was rumored that it shipped with the T2A expansion disc, but both discs that I had (the beta cd and release cd that contained the map) didn't have such a client. It did have some sort of offline demo in the release version if I recall correctly.
 

Obsidian Fire

Sorceror
I was just digging thru my LOL thousands of UO stuff I have on disk and found my GodClient.exe file. looks like it comes from "2.0.8n UOTD/UOR God Client" Shows in the corner.
 

Attachments

  • GodClient Screenie.jpg
    GodClient Screenie.jpg
    46.2 KB · Views: 98

-hash-

Sorceror
I'm not sure how it leaked, but one guess is the now-defunct ftp.owo.com server--maybe it accidentally slipped out on there? My original T2A disc does, however, include a reference to the godclient in the howto.txt file (just talks about unicode font importing).

The 1.23 client has clear indications that it includes some level of god client functionality, though.

In fact, upon reviewing the 7.0.18.0 client, it contains some highly unusual references as well, albeit to a lesser extent. The versioning, for example, had once been completely removed from normal clients; yet, the latest client contains hints to it that it normally shouldn't. My guess is the UO team upgraded their code and when doing so included a bunch of old stuff in the main repo that normally would have been included only in select versions. It includes some manner of reference to all packets between 0x3E-0x45, 0x57, 0x92, 0x94...these would be your update *mul packets.

We shall see...
 

-hash-

Sorceror
Block Sizes / Version Order:
Artwork
0x10000 blocks​
Tiledata
0x400 blocks​
Item Anim
0x800 blocks​
Hues
0x80 blocks​
Regions
0x2000 blocks​
Skills
0x32 blocks​
Multis
0x1ff6 blocks​
Terrain
0x60000 blocks (=768 * 512)​
Statics
0x60000 blocks (=768 * 512)​

Length: 0xD4CA8

For the following files, the "extra" field in their corresponding index is the version number:
Artwork, Skills, Multis, Statics

For the following files, the "header" or "group header" is the version number:
Tiledata, Animdata, Hues, Terrain

The regions file format is attached.
Relevant packet formats are attached.

These formats are from the god client era. I'm sure modern clients have changed some of the packets (like tiledata). I'm investigating that.

VersionTable.cs shows how the version table/tree/whatever is calculated.

The "uint[] versions" used to initialize the table is the collection of versions listed above. So, you'd have something like:

Code:
uint[] versions = new uint[0xD4CA8];

// Load the versions in the order listed...

VersionTable vt = new VersionTable(versions);

For files that have a number of entries that is less than the size listed above, the value 0xFFFFFFFF is used for the missing entry's version.

packets.zip contains formats for:

0x0A - Edit
0x0C - Tiledata
0x3E - Versions
0x3F - Update Statics
0x40 - Update Terrain
0x41 - Update Tiledata
0x42 - Update Art
0x43 - Update Anim
0x44 - Update Hues
0x45 - Ver OK
0x46 - New Artwork
0x47 - New Terrain
0x48 - New Anim
0x49 - New Hues
0x4A - Destroy Artwork
0x4B - Check Version
0x57 - Update Regions
0x58 - New Region
0x5C - Restart Ver
0x61 - Destroy Static
0x62 - Move Static
0x92 - Update Multi
0x94 - Update Skill

I'm working on making more readable documents (lol...), but I'm posting this for now to get it out of my head. So, consider this a preamble of sorts I suppose.
 

Attachments

  • packets.zip
    6.3 KB · Views: 18
  • regions.txt
    459 bytes · Views: 12
  • VersionTable.cs
    1.7 KB · Views: 16

Praxiiz

Sorceror
This is really great information, and very helpful. Thank you for providing it, and keep it up! As I continue to work my way through the client, this really helps to fill in the gaps.

Lately, I've been working on the grid system(s). I've decided that I am going to provide both functionality for the grid using the in-game items, and also with a grid overlay (both will be optional to use). The in-game items will not be actual items, as they will reside on the client only, but they will be targetable and useful for other targeting commands. I will setup an API that can be used server side to control them.

For the overlay, I have had good success with it so far. I've been able to successfully hook the the client to get whatever screen resolution it is using. I've been able to hook the offset as well. What this means is that you can move the window around (both the actual window and the play area) and the overlay will stay fixed on it. Its completely transparent except the grid lines and a small window with extra commands. (Which isn't really implemented yet) I'm using a multimonitor setup which works fine as well. I've been mostly focused on the hooking and getting the overlay to behave as if it were actually a part of the window.

The overlay grid will either be implemented as a plugin module with a button on the Ultima Live tab to bring it up, or it will be part of the stock editor with a button to bring it up. The nice thing about this system is that I should be able to put handles on the map vertices as part of the overlay form. If I can do this, then I could make it possible to click and drag the height of the map.

Here's an early sample of the grid (with an ugly placeholder for the toolbox).

 

-hash-

Sorceror
Here's some more data...

This zip contains my file formats for:

animdata.mul
animinfo.mul
anim*.mul
artdim.mul
art.mul
books.mul
multimap.rle
cliloc (old version and megacliloc version)
fonts.mul
gumpart.mul
hues.mul
lights.mul
multi.mul
palette.mul
radarcol.mul
regions.mul
restypes.mul
sjis2uni.mul
skillgrp.mul
skills.mul
sound.mul
speech.mul
static*.mul
map*.mul
texmaps.mul
tiledata.mul (old and 7.0.18.0 versions)
time.mul
unifont*.mul
verdata.mul

I've just hammered them out based on my source code, so I hope I didn't mistype something :p

The point was these include the proper version references, but they also include bits and pieces probably not seen elsewhere.

Edit:

While I can take credit for my packet definitions, many of the file formats presented here are based on formats from Krrios, Cironian, Steve Dang, Sir Valgriz, Tim Walters, Alazane, and Batlin. There are, however, a few new files and data pieces in old files that come from my work, and I'll vouch for the validity of all included formats :p

Edit #2:

Added the new multi.mul format (exact same except Flags is a 64-bit integer now).
 

Attachments

  • files.zip
    9.5 KB · Views: 17
  • multis-7.0.18.0.txt
    407 bytes · Views: 10

Pure Insanity

Sorceror
Perhaps store all edits on a stack, and only save it when there is something on the stack. Thus only having to spend the time on saving the mul files, when they're actually changed. Of course then I would recommend doing the directory outside of the Saves directory somewhere.
 

Praxiiz

Sorceror
I am working on the next release. I've implemented a standard CRC algorithm (Fletcher16) on both client and server end. Hash Files will no longer be needed on the server and client ends.

I have a proof of concept done for multiple maps. I'm still working on improving the save times further. (There was a slight improvement now that the hashes are gone.)

I've been considering putting the client files in a folder outside the saves folder as Pure Insanity suggests. My only reservation is that shards that want to use the system as a Live System instead of just a map editor will be unable to revert to a specific save and have the world match the save.

The more I think about it, the more I think I need a system similar to SVN for maps. If I built such a system, I would of course have a command that would save the actual client files out to a folder for those using the system as a pure map editor.

Theoretically I could implement the capability to add a map while the server and client are running. This would let map makers create a blank map and begin working on it without rebooting the server or modifying any scripts. Anyone wanting to help build the map could join them and changes would automatically be streamed to them. Its not practical at this time, however, because each new map would add a significant amount of time to the world save. It looks like the world save time will be top priority for now.
 

Praxiiz

Sorceror
I think what Pure Insanity suggested makes the most sense. After further consideration, I think the plan will be as follows:
There will be a change file for each save (only if there are changes on the maps/statics) that is put in a separate folder from the regular save structure.
When the world loads, it will go through the folder and apply each save file to the current map states.
There will be a command to export the full set of client files to disk including the applied changes. This will allow admins to condense the changes from time to time when server loading time gets too high.

After I get this system working, I will concentrate on adding the functionality to support additional maps and perhaps even adding them while the server is running.
 

Pure Insanity

Sorceror
Perhaps make it universal. Allow the server to easily switch between saving it in the saves folder (and with every save), or by saving only after so many changes are on the stack. To cut back on save time. The latter option would be like what you said, for shards that are doing some heavy map editing. The first option would make sense if a shard was to use this system to devise a way of letting players change the map in some way or a form, then it would make sense to save the mul files with every save.

I'm mostly interested in the additional maps. Seems you've got a proof of concept going, to prove to yourself that it could work. But I have a few questions pertaining to this. Will the additional maps also support having their own separate statics? Will there be a way of making the additional maps client side and not have to send every bit of the map over the network. Do you plan on providing easy methods of...maybe copying/cloning a map?

If you have tested the additional maps...did you only test it with the client? Won't we still need to update the RunUO server core to work with the new maps?
 

tass23

Page
Hey Prax, I tried this out and it's really nice, such a great concept. I was wondering though and forgive me for saying so, but building is kind of clunky. Would it be possible to hook this into Pandora's Box?
 

Pure Insanity

Sorceror
It's not clunky, lol. It's just un-finished. Early alpha, basically. It's all server side though. So you could create the commands yourself, new commands, change the commands. Or make a gump in-game to do it all. If you want, you could add the commands to Pandora, believe it lets you add commands to it. If not...it is open source. Just give it time for the commands and world building system as a whole to mature, or start working on it yourself.

Btw, there is one crash bug that I found related to the existing server side commands. If two people try to remove a static at the same time, or edit the terrain. It will cause a crash, but it's an easy enough fix. Do you plan on sticking with the current setup for commands? Plan on changing it any? Would be nice if it played nicely with the [m [area and other commands. Even with distro edits to files to make it play nicely with the modifiers would be worth it, like have it pass the command over to your command methods when it encounters a LandTile or StaticTile or something. A mod like this would make this system far easier to use and more fluid. I believe once this system is a bit more rock solid, the UO community will start seeing some pretty awesome custom maps, as I believe this would allow one to create a map far faster and easier than using something like Dragon or something else to make the map.
 
Top