Here is a couple for you

[Lord_Greywolf]

Here is a couple for you

some hopefully simple but usefull suggestions:

1) if the scripts are cached, also skip the verifying, since they where verified before - will save a few moments on start up of shards with lots of custom scripts

2) a setting to show warning, but not stop because of them?
not sure if possible, but to have it show all warning generated, but still compile and run
this is usefull, because even though your script may have compiled, could have warning in it

3) a setting of "forced cache" - i.e. even if no scripts present or script changes, it will not try to compile, but use the cached dill's instead
also good with #1 above
this can be very usefull, specialy for those scripting on the same machine as the server is, that they know there is problems with the scripts, but need to do a restart
or for remote servers - so if they are hacked, scripts can not be had, just the dll's
and if i remember right, the dll's do not work right if not compiled by that same "runuo.exe" file (had that come up by accident when copying over files from one machine to the other)
so basicaly the dll's do them no good (can copy scripts up - run once to get dll's, set to force cache, and delete scripts, and hacker safe)

4) maybe some backwards compatability
i.e. you change the way something is called
keep old method also, setting it up to do the new way of calling from it
(can place in warniong about it being an older method also maybe? - work great with #2 above)
I know now - this may be hard to put in for older stuff, but at least for newer stuff might be good

5) have a way to modify the "time out" when compiling
i.e. it is a default to about 9 or 10 minutes i believe
well testservers many times do not have the power to compile the scripts up real fast, and hitting that time limit happens a lot - specialy if some windows/antivirus/etc function kicks in while it is going on, and steals 2 minutes from it

6) not sure if possible - but a "quick compile" setting
it would stop before ferivy and say if good or bad
what this does is use the cache and try compiling only new or changed scripts in
this would be usefull for quick script checking to see if any "syntax" errors, etc
but can not be used "live" because was not a full compiling, and other parts may not interact fully with it, until full compiling

thanks fro looking at these,
hope you do not laugh to hard at them
and maybe use 1 or 2 of them

 

[Greystar]

some hopefully simple but usefull suggestions:

1) if the scripts are cached, also skip the verifying, since they where verified before - will save a few moments on start up of shards with lots of custom scripts

2) a setting to show warning, but not stop because of them?
not sure if possible, but to have it show all warning generated, but still compile and run
this is usefull, because even though your script may have compiled, could have warning in it

3) a setting of "forced cache" - i.e. even if no scripts present or script changes, it will not try to compile, but use the cached dill's instead
also good with #1 above
this can be very usefull, specialy for those scripting on the same machine as the server is, that they know there is problems with the scripts, but need to do a restart
or for remote servers - so if they are hacked, scripts can not be had, just the dll's
and if i remember right, the dll's do not work right if not compiled by that same "runuo.exe" file (had that come up by accident when copying over files from one machine to the other)
so basicaly the dll's do them no good (can copy scripts up - run once to get dll's, set to force cache, and delete scripts, and hacker safe)

4) maybe some backwards compatability
i.e. you change the way something is called
keep old method also, setting it up to do the new way of calling from it
(can place in warniong about it being an older method also maybe? - work great with #2 above)
I know now - this may be hard to put in for older stuff, but at least for newer stuff might be good

5) have a way to modify the "time out" when compiling
i.e. it is a default to about 9 or 10 minutes i believe
well testservers many times do not have the power to compile the scripts up real fast, and hitting that time limit happens a lot - specialy if some windows/antivirus/etc function kicks in while it is going on, and steals 2 minutes from it

6) not sure if possible - but a "quick compile" setting
it would stop before ferivy and say if good or bad
what this does is use the cache and try compiling only new or changed scripts in
this would be usefull for quick script checking to see if any "syntax" errors, etc
but can not be used "live" because was not a full compiling, and other parts may not interact fully with it, until full compiling

thanks fro looking at these,
hope you do not laugh to hard at them
and maybe use 1 or 2 of them

I like #2 the other's just require patients.

 

[Lord_Greywolf]

#5 does not deal with patients, it deals with limiting how much you can compile because of the .net "running out of time"

and as for patients, many deal with players patients which we can not controll lol
but it is also as i noted out on a couple, also for security reason too
so in case of hacking, some one else can not use your hord work for themselves

and #3 has nothing to do with patients
you have a crash or restart, and a few scripts in there not completed yet, so it will not compile
the shard will be right back up
rememebr many many shard owners do not have multiple machines to work with
so testing & running are done on same system
(i used to fall into that catagory)

 

[Lokai]

@Lord_Greywolf

Are you using my new Versioning system?

Not sure if you saw the latest downloads. I have versions for 1.0, 2.0 SVN, and 2.0 RC2. If you need one for another specific SVN level, I can make one for you also.

The new LV system allows you to use .CS extensions, and gives you up to 5 folders to place your new custom scripts. New scripts can compile separately, so cached content can load very fast.

 

[Lord_Greywolf]

i am all ready using a modified one of the WB system that includes others as well

if you are tralking about # 4 & 5

#4 is like when they changed the spell system around, to make it so the old ones will still work, just issue warnings to update

and for #5
can only ad so much into the other ones
if stuff is aded into crafting menues, loot drops, etc, or new champ spawns with new monsters, etc -- then they have to be in "main" section
and it can take a while to compile then

 

[Lokai]

True.

I was just wondering, cause I put ALL my customs in the various LV folders, and it really cuts down on compile time for the main folders.

 

[Lord_Greywolf]

unfortunitly - many of my customs also drop as loot (or at least the tools do - and if the tool does, then needs to be in main section also) so they have to be in main section

and like i sai - i have many custom champion spawns, and then those monsters have to be in main section also

but running a 2.2 ghz machine with 2 gig ram - i have it down to ajust under 7 minutes now for the time it takes to compile main section
if nothing else is running (i.e. it get 100% cpu for it)
but if i have something else running to - so it bounces around about 50% give or take - sometimes it goes over the limit and runs out of time

earlier today, i just shave off a big chuck by changing some stuff around, an what used to be 175k memory (watchiong csc go) an them jumping to 184 an being finished is now down to 142k and 151k and about 6 minutes only - so that will help too

but i know many many people have slower machines to work on, and being able to adjust that time would help alot

special;y if they are running in debug moe - boy then does it take longer to compile

 

[Arkryal]

Edit: Old post I know, but it was on the front of the topic page, so please don't flame for my bump.

While the following doesn't really address the changes mentioned, a 7 minute startup is obscene. Even in a fully spawned world with a few dozen custom scripts, my server load time is about 25 seconds. And that's on my old server: 1.2Ghz with 1Gb RAM. That system is dedicated to running servers, so there's not much competing for resources, and much of my speed can be attributed to that, but even still... Here's what I do.

Let's begin by altering your process priority: Free, Easy, Very effective

Right click the shortcut to the RunUO server, and choose properties.
Change the target to:
C:\Windows\System32\cmd.exe /c start /high C:\RunUO\RunUO.exe
and "Start in" to this:
C:\Windows\System32

That should have doubled your speed.

IM software will hijack your thread priority, so disable them or the setting will only last a few minutes. This should massively reduce world saves as well.

Allocated Memory Heaps: Free, not too hard, a little effective.

Many people advocate allotting memory with a heapsize argument from the prompt (XP and Vista only), but the server doesn't require too much, so the results from that are negligable. This can't be done if starting in a high priority mode from a shortcut (you'd allocate memory to cmd.exe, not server), so you'd have to write a batch file for it. Not really worth your time unless you really need to squeeze every last bit of preformance out of it.

RAMDISK Execution: Free, pain in the ass, very effective in some circumstances

Load your server to a RAM Drive on system startup, and execute from there to reduce disk access times. Especially useful if your sending server stats to a website in realtime, accessing XML files from within scripts etc, as it completely eliminates the disk read/write latency. Don't forget, any changes made have to be copied back to the hard disk before shutdown or they will be lost. Batchfiles on startup and shutdown can do that for you easily. Still, I tend to forget to copy back my modifications, and that's not an option here.

SolidState Flash: May cost a bit, easy, effective for busy servers with lots of people

Alternatively, run your server from flash memory if you think disk access is the bottleneck (not likely, but try it). This only works from quality flash memory, not the 1GB for $1.99 USB drives. USB drives will work well for this if they're FAST, so look at reviews before getting one if you go this route. Cheap ones never cut it, but if you've got a good one it's worth a try. I like the Kingston HyperX drives. They're a bit pricey for the capacity, but ungodly fast in the world of USB flash memory. The real advantage to this shows when you have 50+ players on your shard at a time. Also great for running fast web servers. If you've got deep pockets, SSDs are better still, but at $300 for 30GB, the USB comes close enough.

Hope that helps.

 

[shrundowr]

SolidState Flash: May cost a bit, easy, effective for busy servers with lots of people

Alternatively, run your server from flash memory if you think disk access is the bottleneck (not likely, but try it). This only works from quality flash memory, not the 1GB for $1.99 USB drives. USB drives will work well for this if they're FAST, so look at reviews before getting one if you go this route. Cheap ones never cut it, but if you've got a good one it's worth a try. I like the Kingston HyperX drives. They're a bit pricey for the capacity, but ungodly fast in the world of USB flash memory. The real advantage to this shows when you have 50+ players on your shard at a time. Also great for running fast web servers. If you've got deep pockets, SSDs are better still, but at $300 for 30GB, the USB comes close enough.

Hope that helps.

and the average shard would go way past the average write cycle limit in a month or two on even the best flash storage...

best bet is high speed scsi drives. (like the 15K RPM ones UOGamers uses) But I really doubt disc access speed will be the biggest bottleneck...

 

[Arkryal]

and the average shard would go way past the average write cycle limit in a month or two on even the best flash storage...

Two years ago, I'd have agreed with that statement, but the write limitations of flash memory have gone up by a factor of 1000 since devices like USB drives became commonplace. They now utilize many of the same error checking methods as commercial SSDs. Essentially, they track the default bit-state by arranging each written bit in a way that it's two nearest neighbors will serve the dual funcion of parity checking. While this does require more space, modern flash usually has about 30% more room than the OS registers. So if the write limit is exceeded, it still knows the value of the bit that should be there. You would have to overwrite that 30% of the disk in excess of 100,000 - 300,000 times before data loss could occur.

Also, old flash memory used to read/write each bit from the same location. Newer memory is smarter. If there is free room on the drive (it requires about 15% free) it will attampt to move clusters of frequently accessed bits to infrequently used areas of the physical disk.

Also worth noting is the majority of changes that occur on a server can take place in RAM and changes don't necissicarily need to be written back to the physical disk, or are only saved periodically. This ofcouse depends on how the server is written. I haven't even looked at the RunUO Source, so I may be speaking beyond my level of experience. Basically, Servers built for speed with a large capactity for simutanious users tend to opporate in system memory most of the time. This means they take a ton of RAM and a dedicated server is usually necessisary. Servers complied for smaller users bases or lower system requirements would have frequent disk writes to ensure data integrity. I suspect RunUO is the latter, favoring security and and lower requirements over fast speed with high requirements. But that's just speculation on my part, if anyone more familiar with server source would like to correct me, I welcome their input.

Also, while I love my SCSI drive, a 10kRPM SATA II will out benchmark it unless your RAID them. While the SCSI has a 1.5x faster rotation speed, SATA has a 2x greater bit density. That means it (potentially) reads twice as many bits per rotation, even while moving at 2/3 the rotational speed. This assumes bits are arranged contiguously. So a quality SATA drive should theoretically see about a 13% increase in speeds. SCSI however has othe driver controls allowing for on-the-fly defragmentation, error checking etc. SATA can do these things, but typically with greater system overhead. Both ofcourse require software to do this, but most packages are under $100 now, and there's always a torrent... For my money, it's a wash. They breakeven over standard usage, so save a few hundred bucks and go with SATA.

 

[shrundowr]

Two years ago, I'd have agreed with that statement,

I'm talking about today's flash storage

but the write limitations of flash memory have gone up by a factor of 1000 since devices like USB drives became commonplace.

thats not saying much

You would have to overwrite that 30% of the disk in excess of 100,000 - 300,000 times before data loss could occur.

World saves are set for every 15 minutes by default in runuo. Thats 96 overwrites per day.

Also, old flash memory used to read/write each bit from the same location. Newer memory is smarter. If there is free room on the drive (it requires about 15% free) it will attampt to move clusters of frequently accessed bits to infrequently used areas of the physical disk.

It's called wear leveling. It doesn't help much overall.

Basically, Servers built for speed with a large capactity for simutanious users tend to opporate in system memory most of the time. This means they take a ton of RAM and a dedicated server is usually necessisary. Servers complied for smaller users bases or lower system requirements would have frequent disk writes to ensure data integrity. I suspect RunUO is the latter, favoring security and and lower requirements over fast speed with high requirements. But that's just speculation on my part, if anyone more familiar with server source would like to correct me, I welcome their input.

Fairly certain the uogamers shards use runuo, have extremely large playerbases and extremely high end hardware. I really think it has optimizations for both low end hardware and playerbases and high end hardware with large playerbases instead of biasing itself too much toward one or the other.

 

[Jeff]

SATA has a 2x greater bit density. That means it (potentially) reads twice as many bits per rotation, even while moving at 2/3 the rotational speed. .

All SCSI drives made at the same time as a SATA II drive have the same bit density.

This assumes bits are arranged contiguously..

What a stupid statement.... of course they are contiguous.....

So a quality SATA drive should theoretically see about a 13% increase in speeds.

Incorrect, the reason its faster is NCQ (don't know what it means, look it up)

SCSI however has othe driver controls allowing for on-the-fly defragmentation, error checking etc. SATA can do these things, but typically with greater system overhead.

This isnt software, this is hardware....

To be honest, I would only go with SATA II because of the money, and the fact you do not need a 3rd party controller. Othere than this, SCSI is always faster because everything about SCSI is hardware driven. It is a tried and true technology that has rarely been beaten or discredited. The other fact about SCSI drives is they are Enterprise Drives (What does this mean..... It means they have been through more QA then the average drive and are built with the intention to be beaten on at full load 24/7)

 

[Arkryal]

96 writes to a sector per day with a limitation of 300,000 writes yields about 10 years of usage, so I say it's a non-issue.

However, I will concede on the other points, you've definitely done your homework. Good to see people here with a real background and enthusiasm for technology. I usually have to spell everything out, it's good to see some people here just "get it" without further explanation.

You're right about wear leveling, it has a maximum potential of prolonging the life of about 15% of the media, doesn't really do much. If you'd push it to it's limits anyway, 15% is nothing, if you wouldn't it doesn't matter. It really is a marketing word more than a technology, so I'll agree with that.

What a stupid statement.... of course they are contiguous.....

Okay, you got me. Bits within a sector are, sectors on a disk may not be, but it doesn't read sectors in one rotation, it reads bits. Logical clusterfuck on my part. Statement retracted.

This isnt software, this is hardware....

Yes, the controller handles much of the basic functions where as SATA does not, so SATA compensates in software resulting in the higher system overhead I mentioned. That isn't to say either have disk keeping routines that function totally independent of software, however, a server with a SCSI array is likely running linux and support exists inherently there. I should have elaborated, but.... okay, I'll shut up.

 

[Jeff]

96 writes to a sector per day with a limitation of 300,000 writes yields about 10 years of usage, so I say it's a non-issue.

However, I will concede on the other points, you've definitely done your homework. Good to see people here with a real background and enthusiasm for technology. I usually have to spell everything out, it's good to see some people here just "get it" without further explanation.

You're right about wear leveling, it has a maximum potential of prolonging the life of about 15% of the media, doesn't really do much. If you'd push it to it's limits anyway, 15% is nothing, if you wouldn't it doesn't matter. It really is a marketing word more than a technology, so I'll agree with that.

I use to do data recovery, worked in a cleanroom for 3 years, so ya, I know more then the average joe. It's not a marketing word, they really test the crap out of these drives. One thingy the do to every drive is run it in a 110 deg room at full force for 12 hours, enterprise drives recieve 48 hour tests. From all my years of data recovery, rarely did I see SCSI drives, and when I did, they were wrecked and unrecoverable... SATA was second, and PATA was 1st as far as recoveries went....SCSI is 1000% more reliable imo.

 

[Arkryal]

I was referring to wear leveling on solidstate media. I have no background in mechanical disks, so I can't say anything about that. In solidstate, it's a joke. The gains are almost significant with massive arrays of drives. With SSDs you would typically just RAID5 it. On the industry level the cost is acceptable. In home, a little overkill.

 

[shrundowr]

96 writes to a sector per day with a limitation of 300,000 writes yields about 10 years of usage, so I say it's a non-issue.

World saves can take up quite a bit, and with the os and other stuff taking up room also, on a smallish (1-2 gig) bit of flash memory that 10 years starts to shrink.