RunUO Community
This is a sample guest message. Register a free account today to become a member! Once signed in, you'll be able to participate on this site by adding your own topics and posts, as well as connect with other members through your own private inbox!
Save core modification!!It's possible working on it..
- Thread starter Alis
- Start date
Isn't that a bad word on these forums?noobie said:
*I* believe a save modification that will make the times better would be good AND relatively 'easy' to implement. For what she's trying to do, it sounds like it might take a few mods, but if she (we) can pull it off, all the better for the community, and maybe the Devs can take a look after it's completed, do some stress tests and see about putting the changes in.
I believe, what she is trying to do is a 'simple' bool value that is updated when the item (mobile) is changed. The variable is easy, of course, but figuring out a way to update that variable whenever ANYTHING is changed on that item is going to be the hard part. A big chunk of these things could be taken care of with a change to the InvalidateProperties method, but people tend to leave that one out with their custom scripts. I havn't delved into the Core enough to know if there is a lower level call that is always called when something is changed, but there might be? Would just need to throw the update call in that method as well.
I didnt say it is impossible, it was:
ok it is "impossible" without modifying "every single script"
there is not such a method that is called for every single change..
my suggestion for him to work on runtime-save which is better than this and it seems much more "possible" with much less work..
ok it is "impossible" without modifying "every single script"
there is not such a method that is called for every single change..
my suggestion for him to work on runtime-save which is better than this and it seems much more "possible" with much less work..
Zippy
Im aware the major change(modification) is not practical and cannot be implemented.Therefore im seeking solution with simple approach.That will not cost too much.
What im thinking to intercept the base class of mobile,item's,and guild etc.. which will be effective for all derived class.(Of course regeneration of server.exe needed)
Also intercept the point where change request is started, like as ; listener object of server that client event generation will allway's need to be pass through; therefore the atacked point's will be limited to a few object modification.
If we look at the archive site it will request object from universe container and after getting object it will serilize and output; at that point we will insert a very short logic that after obtaining object it will going to ask the object if it is changed or not.If it is not changed simply it will skip this object and will request the next object from the universe container.The change diffrence will be output to different file.The archive will behave to full backup within certain interval just skipping testing object change; in the recovery phase we are not going to do anything except after loading rescent full backup and load the diffrence files consequtively.This might not be costly.
noobie
I asume archive is iterativly getting objects from universe container using an Iterator therefore whenever he get a reffrence of an object and serilize an output to archive file and will ask next object from the universe container.Therefore after obtaining object from universe container even it is changed will not be disturb our mission.Assume that we freeze the system and made archive then unfreeze the system again and let the object be changed it is going to be same scenerio with our case. Asume that the archive will be completed only for this object and start with next object.
Im aware there will be some inconsistent case will be occured for those objects be archived and unarchived, the snapshot of the system at that moment will not reflect the real case.When we restore system from full archive and increments we will not achive the recent state of object that will be archived early and changed again till incremental archive is completed.But according to me this will rarely happend because we are talking about system crash and restoring.Also this can be acceptible because the object early archived has very limited possibility of occur.Even it happens it will loose the very recent change.
If we consider the current archive ; this scenerio cost more dramatically since you are going to loose last 30 min change in crash case(that needs to be restored, current user has a chance to complain about lost modifications on their objects).
For this reason I dont agree to use a secondary hashtable or so.What I am telling the archive can be started in background with in another thread(having lower priority) can run similtanusly and freezing is not necceserry.There is nothing to change but removing freezing process and just start archive in scheduled interval (in other word's nothing is needed to be changed but the operation flow going to be modifed).
Alis said:
ok, there is not any problem with the idea, but the problems are more about programming design.
lets say there are two threads : SaveThread, GameThread
while it is saved, if you use some code like this :
foreach(Item item in World.Items.Values)
{
//save
}
you cant add/remove any item to the world till that loop finishes, otherwise it will throw an exception..(you can not modify any collection while some iterator is running on it ; see hints)
if you dont use a customize hashtable, in the runtime during save, how would you handle new items that is created? or removed?? simply, you cant..
so, if you wanna spend your time, at least you should spend in the right direction..
Yeah exactly like what A_Li_N said is correct the account how matter much it is it will be checked as changed().The time will not extend on the save ..
Noobie
What youa talking about is correct but ; there is another solution as far as I know
but it would be good to see you helping me on this.In other means what a thing done more than 2 people is better then 1 people.Contact me plz
Noobie
What youa talking about is correct but ; there is another solution as far as I know
but it would be good to see you helping me on this.In other means what a thing done more than 2 people is better then 1 people.Contact me plzok then with some search and some asist i diceded to explain step by step what we are going to do.As far as you all know this project is a project open for all people on extracting and asisting the method module we told.
Now firstly we are loonking at a way to stop the freeze done on the world during the save condition.
In the core you must firstly find the World.cs in the c sharp script we are searching the world save class so lets open the script fitsly ;
/
As the colum colored in blue you can see that is the save method of the current runuo save module.
You can see that the save is done in the same thread so in our suggestion we started a new thread with nes hashtable for the save creation.
This is one of the steps we started; the other method which people say it needs allot of modification which i dont beleif is still underconstruction..
The steps will be posted untill it will finnish in few day;s
Now firstly we are loonking at a way to stop the freeze done on the world during the save condition.
In the core you must firstly find the World.cs in the c sharp script we are searching the world save class so lets open the script fitsly ;
/
Code:
***************************************************************************
* World.cs
* -------------------
* begin : May 1, 2002
* copyright : (C) The RunUO Software Team
* email : [email protected]
*
* $Id: World.cs,v 1.4 2005/01/22 04:25:04 krrios Exp $
* $Author: krrios $
* $Date: 2005/01/22 04:25:04 $
*
*
***************************************************************************/
/***************************************************************************
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
***************************************************************************/
using System;
using System.IO;
using System.Collections;
using System.Reflection;
using System.Threading;
using Server;
using Server.Mobiles;
using Server.Accounting;
using Server.Network;
using Server.Guilds;
namespace Server
{
public class World
{
public enum SaveOption
{
Normal,
Threaded
}
public static SaveOption SaveType = SaveOption.Normal;
private static Hashtable m_Mobiles;
private static Hashtable m_Items;
private static bool m_Loading;
private static bool m_Loaded;
private static ArrayList m_DeleteList;
public static bool Saving{ get { return m_Saving; } }
public static bool Loaded{ get { return m_Loaded; } }
public static bool Loading{ get { return m_Loading; } }
private static string mobIdxPath = Path.Combine( "Saves/Mobiles/", "Mobiles.idx" );
private static string mobTdbPath = Path.Combine( "Saves/Mobiles/", "Mobiles.tdb" );
private static string mobBinPath = Path.Combine( "Saves/Mobiles/", "Mobiles.bin" );
private static string itemIdxPath = Path.Combine( "Saves/Items/", "Items.idx" );
private static string itemTdbPath = Path.Combine( "Saves/Items/", "Items.tdb" );
private static string itemBinPath = Path.Combine( "Saves/Items/", "Items.bin" );
private static string regionIdxPath = Path.Combine( "Saves/Regions/", "Regions.idx" );
private static string regionBinPath = Path.Combine( "Saves/Regions/", "Regions.bin" );
private static string guildIdxPath = Path.Combine( "Saves/Guilds/", "Guilds.idx" );
private static string guildBinPath = Path.Combine( "Saves/Guilds/", "Guilds.bin" );
//static World()
//{
// Load();
//}
public static Hashtable Mobiles
{
get
{
return m_Mobiles;
}
}
public static Hashtable Items
{
get
{
return m_Items;
}
}
public static bool OnDelete( object o )
{
if ( !m_Loading )
return true;
m_DeleteList.Add( o );
return false;
}
public static void Broadcast( int hue, bool ascii, string text )
{
Packet p;
if ( ascii )
p = new AsciiMessage( Serial.MinusOne, -1, MessageType.Regular, hue, 3, "System", text );
else
p = new UnicodeMessage( Serial.MinusOne, -1, MessageType.Regular, hue, 3, "ENU", "System", text );
ArrayList list = NetState.Instances;
for ( int i = 0; i < list.Count; ++i )
{
if ( ((NetState)list[i]).Mobile != null )
((NetState)list[i]).Send( p );
}
NetState.FlushAll();
}
public static void Broadcast( int hue, bool ascii, string format, params object[] args )
{
Broadcast( hue, ascii, String.Format( format, args ) );
}
private interface IEntityEntry
Serial Serial{ get; }
int TypeID{ get; }
long Position{ get; }
int Length{ get; }
object Object{ get; }
}
private sealed class RegionEntry : IEntityEntry
{
private Region m_Region;
private long m_Position;
private int m_Length;
public object Object
{
get
{
return m_Region;
}
}
public Serial Serial
{
get
{
return m_Region == null ? 0 : m_Region.UId;
}
}
public int TypeID
{
get
{
return 0;
}
}
public long Position
{
get
{
return m_Position;
}
}
public int Length
{
get
{
return m_Length;
}
}
public RegionEntry( Region r, long pos, int length )
{
m_Region = r;
m_Position = pos;
m_Length = length;
}
}
private sealed class GuildEntry : IEntityEntry
{
private BaseGuild m_Guild;
private long m_Position;
private int m_Length;
public object Object
{
get
{
return m_Guild;
}
}
public Serial Serial
{
get
{
return m_Guild == null ? 0 : m_Guild.Id;
}
}
public int TypeID
{
get
{
return 0;
}
}
public long Position
{
get
{
return m_Position;
}
}
public int Length
{
get
{
return m_Length;
}
}
public GuildEntry( BaseGuild g, long pos, int length )
{
m_Guild = g;
m_Position = pos;
m_Length = length;
}
}
private sealed class ItemEntry : IEntityEntry
{
private Item m_Item;
private int m_TypeID;
private string m_TypeName;
private long m_Position;
private int m_Length;
public object Object
{
get
{
return m_Item;
}
}
public Serial Serial
{
get
{
return m_Item == null ? Serial.MinusOne : m_Item.Serial;
}
}
public int TypeID
{
get
{
return m_TypeID;
}
}
public string TypeName
{
get
{
return m_TypeName;
}
}
public long Position
{
get
{
return m_Position;
}
}
public int Length
{
get
{
return m_Length;
}
}
public ItemEntry( Item item, int typeID, string typeName, long pos, int length )
{
m_Item = item;
m_TypeID = typeID;
m_TypeName = typeName;
m_Position = pos;
m_Length = length;
}
}
private sealed class MobileEntry : IEntityEntry
{
private Mobile m_Mobile;
private int m_TypeID;
private string m_TypeName;
private long m_Position;
private int m_Length;
public object Object
{
get
{
return m_Mobile;
}
}
public Serial Serial
{
get
{
return m_Mobile == null ? Serial.MinusOne : m_Mobile.Serial;
}
}
public int TypeID
{
get
{
return m_TypeID;
}
}
public string TypeName
{
get
{
return m_TypeName;
}
}
public long Position
{
get
{
return m_Position;
}
}
public int Length
{
get
{
return m_Length;
}
}
public MobileEntry( Mobile mobile, int typeID, string typeName, long pos, int length )
{
m_Mobile = mobile;
m_TypeID = typeID;
m_TypeName = typeName;
m_Position = pos;
m_Length = length;
}
}
private static string m_LoadingType;
public static string LoadingType
{
get{ return m_LoadingType; }
}
public static void Load()
{
if ( m_Loaded )
return;
m_Loaded = true;
m_LoadingType = null;
Console.Write( "World: Loading..." );
DateTime start = DateTime.Now;
m_Loading = true;
m_DeleteList = new ArrayList();
int mobileCount = 0, itemCount = 0, guildCount = 0, regionCount = 0;
object[] ctorArgs = new object[1];
Type[] ctorTypes = new Type[1]{ typeof( Serial ) };
ArrayList items = new ArrayList();
ArrayList mobiles = new ArrayList();
ArrayList guilds = new ArrayList();
ArrayList regions = new ArrayList();
if ( File.Exists( mobIdxPath ) && File.Exists( mobTdbPath ) )
{
using ( FileStream idx = new FileStream( mobIdxPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryReader idxReader = new BinaryReader( idx );
using ( FileStream tdb = new FileStream( mobTdbPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryReader tdbReader = new BinaryReader( tdb );
int count = tdbReader.ReadInt32();
ArrayList types = new ArrayList( count );
for ( int i = 0; i < count; ++i )
{
string typeName = tdbReader.ReadString();
Type t = ScriptCompiler.FindTypeByFullName( typeName );
if ( t == null )
{
Console.WriteLine( "failed" );
Console.WriteLine( "Error: Type '{0}' was not found. Delete all of those types? (y/n)", typeName );
if ( Console.ReadLine() == "y" )
{
types.Add( null );
Console.Write( "World: Loading..." );
continue;
}
Console.WriteLine( "Types will not be deleted. An exception will be thrown when you press return" );
throw new Exception( String.Format( "Bad type '{0}'", typeName ) );
}
ConstructorInfo ctor = t.GetConstructor( ctorTypes );
if ( ctor != null )
{
types.Add( new object[]{ ctor, null } );
}
else
{
throw new Exception( String.Format( "Type '{0}' does not have a serialization constructor", t ) );
}
}
mobileCount = idxReader.ReadInt32();
m_Mobiles = new Hashtable( mobileCount );
for ( int i = 0; i < mobileCount; ++i )
{
int typeID = idxReader.ReadInt32();
int serial = idxReader.ReadInt32();
long pos = idxReader.ReadInt64();
int length = idxReader.ReadInt32();
object[] objs = (object[])types[typeID];
if ( objs == null )
continue;
Mobile m = null;
ConstructorInfo ctor = (ConstructorInfo)objs[0];
string typeName = (string)objs[1];
try
{
ctorArgs[0] = (Serial)serial;
m = (Mobile)(ctor.Invoke( ctorArgs ));
}
catch
{
}
if ( m != null )
{
mobiles.Add( new MobileEntry( m, typeID, typeName, pos, length ) );
AddMobile( m );
}
}
tdbReader.Close();
}
idxReader.Close();
}
}
else
{
m_Mobiles = new Hashtable();
}
if ( File.Exists( itemIdxPath ) && File.Exists( itemTdbPath ) )
{
using ( FileStream idx = new FileStream( itemIdxPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryReader idxReader = new BinaryReader( idx );
using ( FileStream tdb = new FileStream( itemTdbPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryReader tdbReader = new BinaryReader( tdb );
int count = tdbReader.ReadInt32();
ArrayList types = new ArrayList( count );
for ( int i = 0; i < count; ++i )
{
string typeName = tdbReader.ReadString();
Type t = ScriptCompiler.FindTypeByFullName( typeName );
if ( t == null )
{
Console.WriteLine( "failed" );
Console.WriteLine( "Error: Type '{0}' was not found. Delete all of those types? (y/n)", typeName );
if ( Console.ReadLine() == "y" )
{
types.Add( null );
Console.Write( "World: Loading..." );
continue;
}
Console.WriteLine( "Types will not be deleted. An exception will be thrown when you press return" );
throw new Exception( String.Format( "Bad type '{0}'", typeName ) );
}
ConstructorInfo ctor = t.GetConstructor( ctorTypes );
if ( ctor != null )
{
types.Add( new object[]{ ctor, typeName } );
}
else
{
throw new Exception( String.Format( "Type '{0}' does not have a serialization constructor", t ) );
}
}
itemCount = idxReader.ReadInt32();
m_Items = new Hashtable( itemCount );
for ( int i = 0; i < itemCount; ++i )
{
int typeID = idxReader.ReadInt32();
int serial = idxReader.ReadInt32();
long pos = idxReader.ReadInt64();
int length = idxReader.ReadInt32();
object[] objs = (object[])types[typeID];
if ( objs == null )
continue;
Item item = null;
ConstructorInfo ctor = (ConstructorInfo)objs[0];
string typeName = (string)objs[1];
try
{
ctorArgs[0] = (Serial)serial;
item = (Item)(ctor.Invoke( ctorArgs ));
}
catch
{
}
if ( item != null )
{
items.Add( new ItemEntry( item, typeID, typeName, pos, length ) );
AddItem( item );
}
}o
tdbReader.Close();
}
idxReader.Close();
}
}
else
{
m_Items = new Hashtable();
}
if ( File.Exists( guildIdxPath ) )
{
using ( FileStream idx = new FileStream( guildIdxPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryReader idxReader = new BinaryReader( idx );
guildCount = idxReader.ReadInt32();
CreateGuildEventArgs createEventArgs = new CreateGuildEventArgs( -1 );
for ( int i = 0; i < guildCount; ++i )
{
idxReader.ReadInt32();//no typeid for guilds
int id = idxReader.ReadInt32();
long pos = idxReader.ReadInt64();
int length = idxReader.ReadInt32();
createEventArgs.Id = id;
BaseGuild guild = EventSink.InvokeCreateGuild( createEventArgs );//new Guild( id );
if ( guild != null )
guilds.Add( new GuildEntry( guild, pos, length ) );
}
idxReader.Close();
}
}
if ( File.Exists( regionIdxPath ) )
{
using ( FileStream idx = new FileStream( regionIdxPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryReader idxReader = new BinaryReader( idx );
regionCount = idxReader.ReadInt32();
for ( int i = 0; i < regionCount; ++i )
{
int typeID = idxReader.ReadInt32();
int serial = idxReader.ReadInt32();
long pos = idxReader.ReadInt64();
int length = idxReader.ReadInt32();
Region r = Region.FindByUId( serial );
if ( r != null )
{
regions.Add( new RegionEntry( r, pos, length ) );
Region.AddRegion( r );
regionCount++;
}
}
idxReader.Close();
}
}
bool failedMobiles = false, failedItems = false, failedGuilds = false, failedRegions = false;
Type failedType = null;
Serial failedSerial = Serial.Zero;
Exception failed = null;
int failedTypeID = 0;
if ( File.Exists( mobBinPath ) )
{
using ( FileStream bin = new FileStream( mobBinPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryFileReader reader = new BinaryFileReader( new BinaryReader( bin ) );
for ( int i = 0; i < mobiles.Count; ++i )
{
MobileEntry entry = (MobileEntry)mobiles[i];
Mobile m = (Mobile)entry.Object;
if ( m != null )
{
reader.Seek( entry.Position, SeekOrigin.Begin );
try
{
m_LoadingType = entry.TypeName;
m.Deserialize( reader );
if ( reader.Position != (entry.Position + entry.Length) )
throw new Exception( String.Format( "***** Bad serialize on {0} *****", m.GetType() ) );
}
catch ( Exception e )
{
mobiles.RemoveAt( i );
failed = e;
failedMobiles = true;
failedType = m.GetType();
failedTypeID = entry.TypeID;
failedSerial = m.Serial;
break;
}
}
}
reader.Close();
}
}
if ( !failedMobiles && File.Exists( itemBinPath ) )
{
using ( FileStream bin = new FileStream( itemBinPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryFileReader reader = new BinaryFileReader( new BinaryReader( bin ) );
for ( int i = 0; i < items.Count; ++i )
{
ItemEntry entry = (ItemEntry)items[i];
Item item = (Item)entry.Object;
if ( item != null )
{
reader.Seek( entry.Position, SeekOrigin.Begin );
try
{
m_LoadingType = entry.TypeName;
item.Deserialize( reader );
if ( reader.Position != (entry.Position + entry.Length) )
throw new Exception( String.Format( "***** Bad serialize on {0} *****", item.GetType() ) );
}
catch ( Exception e )
{
items.RemoveAt( i );
failed = e;
failedItems = true;
failedType = item.GetType();
failedTypeID = entry.TypeID;
failedSerial = item.Serial;
break;
}
}
}
reader.Close();
}
}
m_LoadingType = null;
if ( !failedMobiles && !failedItems && File.Exists( guildBinPath ) )
{
using ( FileStream bin = new FileStream( guildBinPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryFileReader reader = new BinaryFileReader( new BinaryReader( bin ) );
for ( int i = 0; i < guilds.Count; ++i )
{
GuildEntry entry = (GuildEntry)guilds[i];
BaseGuild g = (BaseGuild)entry.Object;
if ( g != null )
{
reader.Seek( entry.Position, SeekOrigin.Begin );
try
{
g.Deserialize( reader );
if ( reader.Position != (entry.Position + entry.Length) )
throw new Exception( String.Format( "***** Bad serialize on Guild {0} *****", g.Id ) );
}
catch ( Exception e )
{
guilds.RemoveAt( i );
failed = e;
failedGuilds = true;
failedType = typeof( BaseGuild );
failedTypeID = g.Id;
failedSerial = g.Id;
break;
}
}
}
reader.Close();
}
}
if ( !failedMobiles && !failedItems && File.Exists( regionBinPath ) )
{
using ( FileStream bin = new FileStream( regionBinPath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
{
BinaryFileReader reader = new BinaryFileReader( new BinaryReader( bin ) );
for ( int i = 0; i <regions.Count; ++i )
{
RegionEntry entry = (RegionEntry)regions[i];
Region r = (Region)entry.Object;
if ( r != null )
{
reader.Seek( entry.Position, SeekOrigin.Begin );
try
{
r.Deserialize( reader );
if ( reader.Position != (entry.Position + entry.Length) )
throw new Exception( String.Format( "***** Bad serialize on {0} *****", r.GetType() ) );
}
catch ( Exception e )
{
regions.RemoveAt( i );
failed = e;
failedRegions = true;
failedType = r.GetType();
failedTypeID = entry.TypeID;
failedSerial = r.UId;
break;
}
}
}
reader.Close();
}
}
if ( failedItems || failedMobiles || failedGuilds || failedRegions )
{
Console.WriteLine( "An error was encountered while loading a saved object" );
Console.WriteLine( " - Type: {0}", failedType );
Console.WriteLine( " - Serial: {0}", failedSerial );
Console.WriteLine( "Delete the object? (y/n)" );
if ( Console.ReadLine() == "y" )
{
if ( failedType != typeof( BaseGuild ) && !failedType.IsSubclassOf( typeof( Region ) ) )
{
Console.WriteLine( "Delete all objects of that type? (y/n)" );
if ( Console.ReadLine() == "y" )
{
if ( failedMobiles )
{
for ( int i = 0; i < mobiles.Count; )
{
if ( ((MobileEntry)mobiles[i]).TypeID == failedTypeID )
mobiles.RemoveAt( i );
else
++i;
}
}
else if ( failedItems )
{
for ( int i = 0; i < items.Count; )
{
if ( ((ItemEntry)items[i]).TypeID == failedTypeID )
items.RemoveAt( i );
else
++i;
}
}
}
}
SaveIndex( mobiles, mobIdxPath );
SaveIndex( items, itemIdxPath );
SaveIndex( guilds, guildIdxPath );
SaveIndex( regions, regionIdxPath );
}
Console.WriteLine( "After pressing return an exception will be thrown and the server will terminate" );
Console.ReadLine();
throw new Exception( String.Format( "Load failed (items={0}, mobiles={1}, guilds={2}, regions={3}, type={4}, serial={5})", failedItems, failedMobiles, failedGuilds, failedRegions, failedType, failedSerial ), failed );
}
EventSink.InvokeWorldLoad();
m_Loading = false;
for ( int i = 0; i < m_DeleteList.Count; ++i )
{
object o = m_DeleteList[i];
if ( o is Item )
((Item)o).Delete();
else if ( o is Mobile )
((Mobile)o).Delete();
}
m_DeleteList.Clear();
foreach ( Item item in m_Items.Values )
{
if ( item.Parent == null )
item.UpdateTotals();
item.ClearProperties();
}
ArrayList list = new ArrayList( m_Mobiles.Values );
foreach ( Mobile m in list )
{
m.ForceRegionReEnter( true );
m.UpdateTotals();
m.ClearProperties();
}
Console.WriteLine( "done ({1} items, {2} mobiles) ({0:F1} seconds)", (DateTime.Now-start).TotalSeconds, m_Items.Count, m_Mobiles.Count );
}
public static void SaveIndex( ArrayList list, string path )
{
if ( !Directory.Exists( "Saves/Mobiles/" ) )
Directory.CreateDirectory( "Saves/Mobiles/" );
if ( !Directory.Exists( "Saves/Items/" ) )
Directory.CreateDirectory( "Saves/Items/" );
if ( !Directory.Exists( "Saves/Guilds/" ) )
Directory.CreateDirectory( "Saves/Guilds/" );
if ( !Directory.Exists( "Saves/Regions/" ) )
Directory.CreateDirectory( "Saves/Regions/" );
using ( FileStream idx = new FileStream( path, FileMode.Create, FileAccess.Write, FileShare.None ) )
{
BinaryWriter idxWriter = new BinaryWriter( idx );
idxWriter.Write( list.Count );
for ( int i = 0; i < list.Count; ++i )
{
IEntityEntry e = (IEntityEntry)list[i];
idxWriter.Write( e.TypeID );
idxWriter.Write( e.Serial );
idxWriter.Write( e.Position );
idxWriter.Write( e.Length );
}
idxWriter.Close();
}
}
internal static int m_Saves;
public static void Save()
{
++m_Saves;
Save( true );
}
private static bool m_MultiProcessor = false;
public static bool MultiProcessor{ get{ return m_MultiProcessor; } set{ m_MultiProcessor = value; } }
public static void Save( bool message )
{
if ( m_Saving || AsyncWriter.ThreadCount > 0 )
return;
NetState.FlushAll();
m_Saving = true;
if ( message )
Broadcast( 0x35, true, "The world is saving, please wait." );
Console.Write( "World: Saving..." );
DateTime startTime = DateTime.Now;
if ( !Directory.Exists( "Saves/Mobiles/" ) )
Directory.CreateDirectory( "Saves/Mobiles/" );
if ( !Directory.Exists( "Saves/Items/" ) )
Directory.CreateDirectory( "Saves/Items/" );
if ( !Directory.Exists( "Saves/Guilds/" ) )
Directory.CreateDirectory( "Saves/Guilds/" );
if ( !Directory.Exists( "Saves/Regions/" ) )
Directory.CreateDirectory( "Saves/Regions/" );
if ( m_MultiProcessor )
{
Thread saveThread = new Thread( new ThreadStart( SaveItems ) );
saveThread.Name = "Item Save Subset";
saveThread.Start();
SaveMobiles();
SaveGuilds();
SaveRegions();
saveThread.Join();
}
else
{
SaveMobiles();
SaveItems();
SaveGuilds();
SaveRegions();
}
//Accounts.Save();
try
{
EventSink.InvokeWorldSave( new WorldSaveEventArgs( message ) );
}
catch ( Exception e )
{
throw new Exception( "World Save event threw an exception. Save failed!", e );
}
//System.GC.Collect();
DateTime endTime = DateTime.Now;
Console.WriteLine( "done in {0:F1} seconds.", (endTime - startTime).TotalSeconds );
if ( message )
Broadcast( 0x35, true, "World save complete. The entire process took {0:F1} seconds.", (endTime - startTime).TotalSeconds );
m_Saving = false;
}
private static void SaveMobiles()
{
ArrayList restock = new ArrayList();
GenericWriter idx;
GenericWriter tdb;
GenericWriter bin;
if ( SaveType == SaveOption.Normal )
{
idx = new BinaryFileWriter( mobIdxPath, false );
tdb = new BinaryFileWriter( mobTdbPath, false );
bin = new BinaryFileWriter( mobBinPath, true );
}
else
{
idx = new AsyncWriter( mobIdxPath, false );
tdb = new AsyncWriter( mobTdbPath, false );
bin = new AsyncWriter( mobBinPath, true );
}
[COLOR="Blue"] idx.Write( (int) m_Mobiles.Count );
foreach ( Mobile m in m_Mobiles.Values )
{
Type t = m.GetType();
long start = bin.Position;
idx.Write( (int) m.m_TypeRef );
idx.Write( (int) m.Serial );
idx.Write( (long) start );
m.Serialize( bin );
idx.Write( (int) (bin.Position - start) );
if ( m is IVendor )
{
if ( ((IVendor)m).LastRestock + ((IVendor)m).RestockDelay < DateTime.Now )
restock.Add( m );
}
m.FreeCache();
}
tdb.Write( (int) m_MobileTypes.Count );
for ( int i = 0; i < m_MobileTypes.Count; ++i )
tdb.Write( ((Type)m_MobileTypes[i]).FullName );
for (int i=0;i<restock.Count;i++)
{
IVendor vend = (IVendor)restock[i];
vend.Restock();
vend.LastRestock = DateTime.Now;
}
idx.Close();
tdb.Close();
bin.Close();
}
internal static ArrayList m_ItemTypes = new ArrayList();
internal static ArrayList m_MobileTypes = new ArrayList();
private static void SaveItems()
{
ArrayList decaying = new ArrayList();
GenericWriter idx;
GenericWriter tdb;
GenericWriter bin;
if ( SaveType == SaveOption.Normal )
{
idx = new BinaryFileWriter( itemIdxPath, false );
tdb = new BinaryFileWriter( itemTdbPath, false );
bin = new BinaryFileWriter( itemBinPath, true );
}
else
{
idx = new AsyncWriter( itemIdxPath, false );
tdb = new AsyncWriter( itemTdbPath, false );
bin = new AsyncWriter( itemBinPath, true );
}
idx.Write( (int) m_Items.Count );
foreach ( Item item in m_Items.Values )
{
if ( item.Decays && item.Parent == null && item.Map != Map.Internal && (item.LastMoved + item.DecayTime) <= DateTime.Now )
decaying.Add( item );
long start = bin.Position;
idx.Write( (int) item.m_TypeRef );
idx.Write( (int) item.Serial );
idx.Write( (long) start );
item.Serialize( bin );
idx.Write( (int) (bin.Position - start) );
item.FreeCache();
}
tdb.Write( (int) m_ItemTypes.Count );
for ( int i = 0; i < m_ItemTypes.Count; ++i )
tdb.Write( ((Type)m_ItemTypes[i]).FullName );
idx.Close();
tdb.Close();
bin.Close();
for ( int i = 0; i < decaying.Count; ++i )
{
Item item = (Item)decaying[i];
if ( item.OnDecay() )
item.Delete();
}
}
private static void SaveGuilds()
{
GenericWriter idx;
GenericWriter bin;
if ( SaveType == SaveOption.Normal )
{
idx = new BinaryFileWriter( guildIdxPath, false );
bin = new BinaryFileWriter( guildBinPath, true );
}
else
{
idx = new AsyncWriter( guildIdxPath, false );
bin = new AsyncWriter( guildBinPath, true );
}
idx.Write( (int) BaseGuild.List.Count );
foreach ( BaseGuild guild in BaseGuild.List.Values )
{
long start = bin.Position;
idx.Write( (int)0 );//guilds have no typeid
idx.Write( (int)guild.Id );
idx.Write( (long)start );
guild.Serialize( bin );
idx.Write( (int) (bin.Position - start) );
}
idx.Close();
bin.Close();
}
private static void SaveRegions()
{
int count = 0;
GenericWriter bin;
if ( SaveType == SaveOption.Normal )
bin = new BinaryFileWriter( regionBinPath, true );
else
bin = new AsyncWriter( regionBinPath, true );
MemoryStream mem = new MemoryStream( 4 + (20*Region.Regions.Count) );
BinaryWriter memIdx = new BinaryWriter( mem );
memIdx.Write( (int)0 );
for ( int i = 0; i < Region.Regions.Count; ++i )
{
Region region = (Region)Region.Regions[i];
if ( region.Saves )
{
++count;
long start = bin.Position;
memIdx.Write( (int)0 );//typeid
memIdx.Write( (int) region.UId );
memIdx.Write( (long) start );
region.Serialize( bin );
memIdx.Write( (int) (bin.Position - start) );
}
}
bin.Close();
memIdx.Seek( 0, SeekOrigin.Begin );
memIdx.Write( (int)count );
if ( SaveType == SaveOption.Threaded )
{
AsyncWriter asyncIdx = new AsyncWriter( regionIdxPath, false );
asyncIdx.MemStream = mem;
asyncIdx.Close();
}
else
{
FileStream fs = new FileStream( regionIdxPath, FileMode.Create, FileAccess.Write, FileShare.None );
mem.WriteTo( fs );
fs.Close();
mem.Close();
}
// mem is closed only in non threaded saves, as its reference is copied to asyncIdx.MemStream
memIdx.Close();
}
public static IEntity FindEntity( Serial serial )
{
if ( serial.IsItem )
{
return (Item)m_Items[serial];
}
else if ( serial.IsMobile )
{
return (Mobile)m_Mobiles[serial];
}
else
{
return null;
}
}
public static Mobile FindMobile( Serial serial )
{
return (Mobile)m_Mobiles[serial];
}
public static void AddMobile( Mobile m )
{
m_Mobiles[m.Serial] = m;
}
public static Item FindItem( Serial serial )
{
return (Item)m_Items[serial];
}
public static void AddItem( Item item )
{
m_Items[item.Serial] = item;
}
public static void RemoveMobile( Mobile m )
{
m_Mobiles.Remove( m.Serial );
}
public static void RemoveItem( Item item )
{
m_Items.Remove( item.Serial );
}
}
}[/COLOR]As the colum colored in blue you can see that is the save method of the current runuo save module.
You can see that the save is done in the same thread so in our suggestion we started a new thread with nes hashtable for the save creation.
This is one of the steps we started; the other method which people say it needs allot of modification which i dont beleif is still underconstruction..
The steps will be posted untill it will finnish in few day;s
Zippy
Razor Creator
I don't see a way you can determine if an object's serialization data has changed without modifying every script that does serialization to call a "MarkObjectChanged();" function or whatever.
And the reason a save which does not "freeze"/stall the entire shard (the game thread) will NEVER work is simple:
1) Save Begins
2) My character is saved, along with it my entire backpack (including a stack of gold in my backpack with amount=1000 and a sword)
3) I sell the sword to some guy (who has not yet been saved) for 500gp.
4) The guy gets saved
5) The shard crashes
6) The shard restarts, loads
7) Where is the sword? Where is the 500gp?
You might be able to get around this specific instance of the problem, but the over all problem remains... parts of the world are out of sync with each other. The whole point of a world save is to recover from unexpected crashes but under a system like this, these crashes would render the save data unusable.
Believe me, I've thought about this a lot. The most effective way to speed up RunUO's current saves would be a "delta" save, but as I've said it would just cause too many problems for RunUO users.
And the reason a save which does not "freeze"/stall the entire shard (the game thread) will NEVER work is simple:
1) Save Begins
2) My character is saved, along with it my entire backpack (including a stack of gold in my backpack with amount=1000 and a sword)
3) I sell the sword to some guy (who has not yet been saved) for 500gp.
4) The guy gets saved
5) The shard crashes
6) The shard restarts, loads
7) Where is the sword? Where is the 500gp?
You might be able to get around this specific instance of the problem, but the over all problem remains... parts of the world are out of sync with each other. The whole point of a world save is to recover from unexpected crashes but under a system like this, these crashes would render the save data unusable.
Believe me, I've thought about this a lot. The most effective way to speed up RunUO's current saves would be a "delta" save, but as I've said it would just cause too many problems for RunUO users.
I'm not extremely versed in this matter, but wouldn't saving to memory be much faster than directly to HDD? Then dump the mem stream to the HDD in the background. This would still create a pause, but that pause would be much less. I know that this has been discussed before, but I just wanted to bring it up again.
Zippy said:
Well thanks zippy for your suggestion but i allready told that
Im aware there will be some inconsistent case will be occured for those objects be archived and unarchived, the snapshot of the system at that moment will not reflect the real case.When we restore system from full archive and increments we will not achive the recent state of object that will be archived early and changed again till incremental archive is completed.But according to me this will rarely happend because we are talking about system crash and restoring.Also this can be acceptible because the object early archived has very limited possibility of occur.Even it happens it will loose the very recent change.
If we consider the current archive ; this scenerio cost more dramatically since you are going to loose last 30 min change in crash case(that needs to be restored, current user has a chance to complain about lost modifications on their objects).
I except what you said but only the trade done between during the save which such only could be affected during a crash could be macimum 10 people suffering such a problem but this is acceptable because it only happens during a crash phase .Its a terminolgy of racing for same resource.i also said this bedore but it is acceptable.
So it is highyly exceptable.
The other save method has no problems for the runuo users .
What i mean both methods are highly recommended and usefull.
Both of them are stable and only (1.st) method has a probality of having such problem the other delta save method has no pronblem in consistent.Zippy i wished that you have done this way a long time ago which would really change allot of things better.
Zippy has a good point there
actually the first problem is easy to solve, but out of sync problem is nothing easy to handle
for delta save issue, if it would be possible to write a generic hash function for serialization info, the rest would be peace of cake
actually, it is possible (not specificly for serialization info but all props) but it wouldnt be an efficient hash function anyway
actually the first problem is easy to solve, but out of sync problem is nothing easy to handle
for delta save issue, if it would be possible to write a generic hash function for serialization info, the rest would be peace of cake
actually, it is possible (not specificly for serialization info but all props) but it wouldnt be an efficient hash function anyway
Zippy said:
I dont see a problem on save crashes. If the shard crashes meanwhile an unfrozen save is being done, just check if save is finished (we can have some flag on disk to determine a succesful save), otherwise is a failed save and you shouldn't use it.
The problem is that it can duplicate objects on valid saves.
1) A sword is saved
2) the sword is sold to an npc for 500 gps
3) the 500gps are saved
What happens? On a restart, we'll have both items.
or
1) Player A have 30K gold
2) Player A is saved
3) player A gives X gold to player B
4) Player B is saved
What happens? Money duped.
Delta saves can be a good approach. Maybe a simple CRC can do the hard job. Let's say we want to save an item:
-We need a new hidden var, a SavedCRC
-On serialize, the item produce an array of bytes that are all those Write().
-Before really saving it, we obtain a CRC value. This should be an unique number depending on the array to be saved.
-Compare new CRC with SavedCRC.
+If equals the item remains unchanged and we can do a delta save.
+If CRC!=SavedCRC then SavedCRC=CRC and save the item as normally
That idea doesn't really change scripts a lot (you don't need to change every de/serialize, just on Core the save manager and the writer & reader). We also must have a complete save every X delta saves, let's say 1 each 12 hours.
Others ideas that can speed up saving times:
1) Do not touch anything. Create a Virtual HD on memory. Save on this disk, and with another program keep polling if a save is completed. Once you have a save, the program should copy the savegame to the real hard disk. The program at start also must copy the last save from disk to virtual for a proper load.
2) A memory buffer enough to keep all the save state. Once on buffer, we can save to disk with another thread.
3) I dont really know how serialize works internally, but it should have a 20MB buffer to save. Once it's full, it should make a blockwrite to disk. its faster to save 20MB as a block rather than 1000 saves of 20KB.
4) Also do a compress to the data on buffer idea 3). On those 20MB of buffer, we'll have a lot of item with many similarities. That means a low entropy, therefore we can compress a lot. Here may happen that compress time+save to disk>raw save to disk, that should be tested.
All those ideas can also be used with delta saves, they work at a different level.
What your talking about is not a efficient technica way the delta saves which im talking about could also use the system that your talking about lkie using delta and using ram save ... which A_Li_N said before but thats not quite effiecnt
What zippy is telling about the not freeze save method is correct but its acceptable 99% its stable and working 1% percent of chance loosing some files.. but this is acceptable the chance of loosing files is way toooo low ..
Mrkrok
You still need good rams and good superconductoer hdd to that but in my method you dont need to waste money on them with 72krpm hdds it will save a shard big like UOhybird in probably maximum 3-4 seconds. ..
What zippy is telling about the not freeze save method is correct but its acceptable 99% its stable and working 1% percent of chance loosing some files.. but this is acceptable the chance of loosing files is way toooo low ..
Mrkrok
You still need good rams and good superconductoer hdd to that but in my method you dont need to waste money on them with 72krpm hdds it will save a shard big like UOhybird in probably maximum 3-4 seconds. ..
brodock
Sorceror
i think what you are trying to do is similar to how Transactios work with databases... i was studying some ways to backup a "living database" and they used a simple trick to do a save without stop working...
let me explain:
they start to write every single database to the files...
while the "saving process" is running, every command and modification are stored in a log file...
hummm i came with something that can improve your idea...
also it's already working and developed...
google for XPrevail
let me explain:
they start to write every single database to the files...
while the "saving process" is running, every command and modification are stored in a log file...
hummm i came with something that can improve your idea...
also it's already working and developed...
google for XPrevail