Save core modification!!It's possible working on it..

[Alis]

Originally Posted by ArteGordon

I have a small suggestion. I dont think that your approach of trying to identify objects as having changed by flagging all property mods is the right way to go. It requires massive script modification, and not just in the core. Also, it is highly likely to miss modifications to serialized properties that dont go through accessors.
Now maybe you are actually trying something else and I'm just assuming incorrectly, but a different approach would be to simply modify the Serialization call so that it doesnt flush its buffers immediately to disk but instead maintains it in memory prior to writing. Once you had fully deserialized an object into a memory buffer, run a checksum on that buffer and compare the checksum to the value from the most recent save to determine if anything had changed.
If the checksum matches, then the serialization is consistent with previous values and you dont have to save, if it doesnt then you have to write it out.
You could even establish the checksums during Deser and maintain them internally until Serialization, updating them accordingly when you delta save.
This requires no mods to anything other than the core Ser/Deser routines.

First'ly your correct that it need's major core modifing but by simple test's taken from the old archive a creating a new archive item we could see the diffrence..

If you look back in the thread i explained that this system is possible so no one can say it's not possible =) But need's alot of work.. It's stable on theoritical way's ..So I used theories to suggest it's stable or not.By the progaming dialog and the scales used in c++ & visualbasic this must be stable..

[ArteGordon]

Originally Posted by Alis

First'ly your correct that it need's major core modifing but by simple test's taken from the old archive a creating a new archive item we could see the diffrence..

If you look back in the thread i explained that this system is possible so no one can say it's not possible =) But need's alot of work.. It's stable on theoritical way's ..So I used theories to suggest it's stable or not.By the progaming dialog and the scales used in c++ & visualbasic this must be stable..

I agree that it is possible, but I dont think that it requires such major core modification beyond the actual Ser/Deser methods and the GenericWriter.

[A_Li_N]

I agree with ArteGordon...I don't think the mods you are trying to accomplish would need to be so massive. You might give his ideas a try and see if you can do some tests.

(What I got from his ideas):
Have the Ser method check the current object being 'saved' against the last save information for that item. If it's changed (or if it doesn't exist), it would save that object. (By object, I mean anything that is saved, like a 'Name' or 'ItemID', etc.

(What *my* suggestion is):
My suggestion (as I told you on MSN Alis ) is to not check all 'objects', but instead simply add a variable that is NOT saved. This variable would be a DateTime. This DateTime would be 'updated' when anything is changed on the Item/Mobile. The Ser method then checks this variable and compares it to the last save DateTime. If the variable is newer (meaning the item/mobile was changed since the last save), the item/mobile is saved. I know this is more information that would be saved, but *I* believe that this would be a lot easier to implement.

But still, if you know what you are wanting to do, just give me a list and I'll get started

[Alis]

Arte ; your correct on not modifyin all the scripts I found out what i need to modify (mobile ; items) these are the basic thing's but in other way what your talking about will slow down the machine mine is diffrent further today i will explain what Im doing.

Save Core 1.0beta will be released by next week =) hope that i will have the support then =)

[noobie]

you cant set the "static" or "unchanged" flag for every item.
how could you possibly do that??

editing all the setters in the whole system? not just the core, but every single item/mobile files??

[Alis]

you cant set the "static" or "unchanged" flag for every item.
how could you possibly do that??

editing all the setters in the whole system? not just the core, but every single item/mobile files??

Yeah this is possible only need's to know the techincal way =) which I know which next weak or 2-3 day if i have enought time i will post it.

YOu don't have to edit evry single object (mobile/item) all the items link to one script all item's link to one script which offers two domain scripts but not all the scripts =)

[ArteGordon]

I'll be interested to see what you come up with. good luck.

[noobie]

I am sorry but as far as I know it is impossible to do it just by modifying Item class

You can do it in Item class for only Item properties but almost any item has its own properties too. You cant control those properties with Item class..

besides, you might not be able to do it even for some Item properties where some child items do not call Base methods in overriding methods..

(same for mobiles)

[Alis]

I am sorry but as far as I know it is impossible to do it just by modifying Item class

You can do it in Item class for only Item properties but almost any item has its own properties too. You cant control those properties with Item class..

besides, you might not be able to do it even for some Item properties where some child items do not call Base methods in overriding methods..

(same for mobiles)

Well you are getting some where... let me express like this may be you will totally understand better ; but you got the point ; ok ;

In object technolgy allmost evry class is generated from a base or abstract class whenever a new inherits or ( extend) this base class it will inherit their difiend method's and properties and if it likes it will overwrite base class method's and add a new properties also.

In some language the overwriting(overloading) a method has some mechanisim that either replace the base method or just reply method request by it's new overloaded method this concept is called "virtual method".When ever base method is typed as virtual and the extended class overload's this method then their is no chance to call base method as you said; but if is not typed as virtual then you can call base method by casting extended method to base method.Then you will be able to call base method even.

Therefore the description of method can be called from the child object .If the method is not overloaded by extended class then you can allway's call the base method.

Our goal is going to be define a property bool ChangeFlag and define three method
-"Void SetChanged()",
-"Void UnsetChanged()",
-"Bool IsChanged()";

In base class like (class Mobile) and never care in the child classes about this stuff.Then we are going tp atack the code sniplet where the action's is conducted;like ([move command) We will call the object SetChanged() method that is going set ChagedFlag property to true.This action will be also intercept in new creation time of item.İtem must be set as changed.

We will also ; atack the archive process which is taking object from universe container and serilize them to write archive file.At that point we will intercept a logic that will check the object with IsChanged () method to differantiate if it is changed or not since recent archive.
If it is changed it will going to be serillized and written to archive then call UnsetChanged() method to reset ChangeFlag to false.

There for no child class will be involved with this flag .This will save the script writer's even know about such a mechanisim is exist.

[ArteGordon]

I was guessing that you were trying something like that, but the question is how are you going to handle the modifications to other properties that might be changed in a class?
If any serialized property changes then in your scheme, wouldnt that require that you set your 'changed' flag?
The example that you give would handle the case of changed location, but you would have to be able to extend this to all other serialized properties for it to work.

[noobie]

hehe, thanx for that OOP info

dont get me wrong, just take this as an advise: you better ask someone more experienced before you spend your time about it.

good luck..

[noobie]

actually, you could focus on your other idea, runtime-save so that save time doesnt matter anymore..

there is already a code snippet for it in the core, you can think about what could be the side effects of doing everyting totally in a different thread..

some hints:
-you cant modify a collection (ex: hashtable) while some iterator is working on it.
the behaviour of World.RemoveItem should be changed a little..

-same problem with World.AddItem
(this is more problematic, you might need another temporary hashtable)

-so you need to write a another customized hashtable class which has internally two hashtable(normal & temp)

-modify World.FindMobile, World.FindItem, Items (it should return customized hashtable),
Mobiles (same with Items: there should be another customized hashtable for it)

-you need to modifiy Add, Remove methods for customized hashtables: see hint 1,2

look into core for more info..

p.m me if you need help with anything..

[Alis]

I was guessing that you were trying something like that, but the question is how are you going to handle the modifications to other properties that might be changed in a class?
If any serialized property changes then in your scheme, wouldnt that require that you set your 'changed' flag?
The example that you give would handle the case of changed location, but you would have to be able to extend this to all other serialized properties for it to work.

I think dealing with individual properties change Is not neccsary.The property/properties change Is initiated in several points;when ever an event occur in the client; the message is broadcast to server.The server will accept this message and initiate several actions.At that point we know the client is going to change.If we insert SetChanged() that point for the client object then we can mark the entire client is change.If neccasisary we can furtherly speciallize about event messsage to flitter out unnecisary change mark.
Therefore it is not neccasary to deal with every indivual properties of any item.We will difrenciet the object's subjet to be archived and need not to be archived.

actually, you could focus on your other idea, runtime-save so that save time doesnt matter anymore..

there is already a code snippet for it in the core, you can think about what could be the side effects of doing everyting totally in a different thread..

some hints:
-you cant modify a collection (ex: hashtable) while some iterator is working on it.
the behaviour of World.RemoveItem should be changed a little..

-same problem with World.AddItem
(this is more problematic, you might need another temporary hashtable)

-so you need to write a another customized hashtable class which has internally two hashtable(normal & temp)

-modify World.FindMobile, World.FindItem, Items (it should return customized hashtable),
Mobiles (same with Items: there should be another customized hashtable for it)

-you need to modifiy Add, Remove methods for customized hashtables: see hint 1,2

look into core for more info..

p.m me if you need help with anything..

I really didn't understand which second thought(idea) your talking about because one of the idea was my dream ; the other was the releif of current archive sys.

[Zippy]

The largest problem with this method is the huge amount of script modifications required... that the problem for any major save change, which is why saves are very unlikely to change in any new RunUO releases.

Otherwise it's a good idea.

[Alis]

Zippy what do you think for a estimation what is the percentage that will give us great save time saved in my suggestion we don't need to modify very much why do we have to ? Is
it the same thing which ArteGordon is talking about ?

[noobie]

Alis, isn't this one of your ideas ?

Originally Posted by Alis

---------------------------New disscusion same topic-------------------------------

For sympthom relief of current archive system I am also wondering why archive and living system is not run concurently.If concern is making identical copy of living system with archive is froze running system and make backup is not a good approach to me .. If we start to archive an object and this object is furtherly changed during backup will not harm our goal since this object will be archived with next cycle.Therfore we can archive universe without froze(with lower priority).This will be appear as unfrozed system(like osi).

So, this is just an approach for you to begin with..

Originally Posted by noobie

actually, you could focus on your other idea, runtime-save so that save time doesnt matter anymore..

there is already a code snippet for it in the core, you can think about what could be the side effects of doing everyting totally in a different thread..

some hints:
-you cant modify a collection (ex: hashtable) while some iterator is working on it.
the behaviour of World.RemoveItem should be changed a little..

-same problem with World.AddItem
(this is more problematic, you might need another temporary hashtable)

-so you need to write a another customized hashtable class which has internally two hashtable(normal & temp)

-modify World.FindMobile, World.FindItem, Items (it should return customized hashtable),
Mobiles (same with Items: there should be another customized hashtable for it)

-you need to modifiy Add, Remove methods for customized hashtables: see hint 1,2

look into core for more info..

p.m me if you need help with anything..

[A_Li_N]

Originally Posted by noobie

impossible

Isn't that a bad word on these forums?

*I* believe a save modification that will make the times better would be good AND relatively 'easy' to implement. For what she's trying to do, it sounds like it might take a few mods, but if she (we) can pull it off, all the better for the community, and maybe the Devs can take a look after it's completed, do some stress tests and see about putting the changes in.

I believe, what she is trying to do is a 'simple' bool value that is updated when the item (mobile) is changed. The variable is easy, of course, but figuring out a way to update that variable whenever ANYTHING is changed on that item is going to be the hard part. A big chunk of these things could be taken care of with a change to the InvalidateProperties method, but people tend to leave that one out with their custom scripts. I havn't delved into the Core enough to know if there is a lower level call that is always called when something is changed, but there might be? Would just need to throw the update call in that method as well.

[noobie]

I didnt say it is impossible, it was:

I am sorry but as far as I know it is impossible to do it just by modifying Item class

ok it is "impossible" without modifying "every single script"

there is not such a method that is called for every single change..


my suggestion for him to work on runtime-save which is better than this and it seems much more "possible" with much less work..

[Alis]

The largest problem with this method is the huge amount of script modifications required... that the problem for any major save change, which is why saves are very unlikely to change in any new RunUO releases.

Otherwise it's a good idea.

Zippy

Im aware the major change(modification) is not practical and cannot be implemented.Therefore im seeking solution with simple approach.That will not cost too much.
What im thinking to intercept the base class of mobile,item's,and guild etc.. which will be effective for all derived class.(Of course regeneration of server.exe needed)

Also intercept the point where change request is started, like as ; listener object of server that client event generation will allway's need to be pass through; therefore the atacked point's will be limited to a few object modification.

If we look at the archive site it will request object from universe container and after getting object it will serilize and output; at that point we will insert a very short logic that after obtaining object it will going to ask the object if it is changed or not.If it is not changed simply it will skip this object and will request the next object from the universe container.The change diffrence will be output to different file.The archive will behave to full backup within certain interval just skipping testing object change; in the recovery phase we are not going to do anything except after loading rescent full backup and load the diffrence files consequtively.This might not be costly.

Alis, isn't this one of your ideas ?


Quote:
Originally Posted by Alis
---------------------------New disscusion same topic-------------------------------

For sympthom relief of current archive system I am also wondering why archive and living system is not run concurently.If concern is making identical copy of living system with archive is froze running system and make backup is not a good approach to me .. If we start to archive an object and this object is furtherly changed during backup will not harm our goal since this object will be archived with next cycle.Therfore we can archive universe without froze(with lower priority).This will be appear as unfrozed system(like osi).


So, this is just an approach for you to begin with..


Quote:
Originally Posted by noobie
actually, you could focus on your other idea, runtime-save so that save time doesnt matter anymore..

there is already a code snippet for it in the core, you can think about what could be the side effects of doing everyting totally in a different thread..

some hints:
-you cant modify a collection (ex: hashtable) while some iterator is working on it.
the behaviour of World.RemoveItem should be changed a little..

-same problem with World.AddItem
(this is more problematic, you might need another temporary hashtable)

-so you need to write a another customized hashtable class which has internally two hashtable(normal & temp)

-modify World.FindMobile, World.FindItem, Items (it should return customized hashtable),
Mobiles (same with Items: there should be another customized hashtable for it)

-you need to modifiy Add, Remove methods for customized hashtables: see hint 1,2

look into core for more info..

p.m me if you need help with anything..

noobie

I asume archive is iterativly getting objects from universe container using an Iterator therefore whenever he get a reffrence of an object and serilize an output to archive file and will ask next object from the universe container.Therefore after obtaining object from universe container even it is changed will not be disturb our mission.Assume that we freeze the system and made archive then unfreeze the system again and let the object be changed it is going to be same scenerio with our case. Asume that the archive will be completed only for this object and start with next object.

Im aware there will be some inconsistent case will be occured for those objects be archived and unarchived, the snapshot of the system at that moment will not reflect the real case.When we restore system from full archive and increments we will not achive the recent state of object that will be archived early and changed again till incremental archive is completed.But according to me this will rarely happend because we are talking about system crash and restoring.Also this can be acceptible because the object early archived has very limited possibility of occur.Even it happens it will loose the very recent change.

If we consider the current archive ; this scenerio cost more dramatically since you are going to loose last 30 min change in crash case(that needs to be restored, current user has a chance to complain about lost modifications on their objects).

For this reason I dont agree to use a secondary hashtable or so.What I am telling the archive can be started in background with in another thread(having lower priority) can run similtanusly and freezing is not necceserry.There is nothing to change but removing freezing process and just start archive in scheduled interval (in other word's nothing is needed to be changed but the operation flow going to be modifed).

[noobie]

hehe, just re-take the advice

those hints are there for a reason

[Alis]

Noobie i guess you did't read whay i posted your advises are not logical

[Quantos]

It would be interesting to see how such a system would perform when over 2000 inactive accounts become deleted when an admin cleans up the shard. Yes, we have had that many inactive accounts cleaned from Demise.

[Alis]

I didnt get what you ment ?