RunUO Community

This is a sample guest message. Register a free account today to become a member! Once signed in, you'll be able to participate on this site by adding your own topics and posts, as well as connect with other members through your own private inbox!

Errors and a crash

Saumane

Wanderer
Errors and a crash

Sometimes i get this error/warning/what ever it is:
Code:
System.Net.Sockets.SocketException: The socket is not connected
  at System.Net.Sockets.Socket.Shutdown (SocketShutdown how) [0x00000] 
  at Server.Network.NetState.Dispose (Boolean flush) [0x00000]
(Usually when a client disconnects)

And sometimes my server crashes, and the error is "Too many open files"(i will give more specific error report when the shard crashes next time), and when i restart my server, half of the folders inside the saves folder has been deleted.

Any fixes?
 

ClanCC

Wanderer
Saumane;772140 said:
Sometimes i get this error/warning/what ever it is:
Code:
System.Net.Sockets.SocketException: The socket is not connected
  at System.Net.Sockets.Socket.Shutdown (SocketShutdown how) [0x00000] 
  at Server.Network.NetState.Dispose (Boolean flush) [0x00000]
(Usually when a client disconnects)
Did you edit ServerList.cs?

If you did, did you changed the public static readonly string Address?

I got that error before, when i changed public static readonly string Address.

Also, which RunUO are you using? SVN or official release?
 

ClanCC

Wanderer
Saumane;772415 said:
I have not edited the public static readonly string Address.
I'm using RUO2 SVN.
hm, did u do any modifications to any scripts or added any scripts?
 

Saumane

Wanderer
Code:
Login: **.***.***.**: Valid credentials for '*****'
System.Net.Sockets.SocketException: Too many open files
  at System.Net.Sockets.Socket..ctor (AddressFamily family, SocketType type, ProtocolType proto) [0x00000] 
  at Server.Network.SocketPool.AcquireSocket () [0x00000] 
  at Server.Network.Listener.OnAccept (IAsyncResult asyncResult) [0x00000] 
Client: **.***.***.**: Disconnected. [11 Online] [*****]
Core: Using dual save strategy
World: Saving...Error:
Error:
SysSystem.IO.IOExcetem.IO.IOExcepption: tion: Too many openTo files
  at System.IO.FileStream..ctor (So many open files
  at System.ystem.IO.FileStream..ctor (System.StrString naming ne, FileMoame, d
e mode, FiFileMode mode, FileAcleAccess accceesss as, FileShare shacrce, Iess, 
FileShare nsth3a2r buffee,rSize, B oIoleannt3 2 buffanonymous, FileeOrSize, Booleapn tanonymions optoiuons) [0x00000]s ,
  F iat System.IO.FlielOepStitream..ctoonrs options) [0x00000] 
  at System.IO.FileStream..ctor (System.String name, FileMode mode, FileAccess access, FileShare share) [0x00000] 
  at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
  at Server.BinaryFileWriter..ctor (System.String filename, Boolean prefixStr) [0x00000] 
  at Server.StandardSaveStrategy.SaveMobiles (Server.SaveMetrics metrics) [0x00000] 
  at Server.DualSaveStrategy.Save (Server.SaveMetrics metrics) [0x00000] 
  at Server.World.Save (Boolean message) [0x00000] 
  at Server.World.Save () [0x00000] 
  at Server.Misc.AutoSave.Save (System.String name, FileMode mode, FileAccess access, FileShare share) [0x00000] 
  at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
  at Ser () [0x00000] 
  at Server.Misc.AutoSave.OnTick () [0x00000] 
  at Server.Timer.Slice () [0x00000] 
  at Server.Core.Main (System.String[] args) [0x00000]ver.BinaryFileWriter..ctor (System.String filename, Bool 
ean prefixStr) [0x00000] 
  at Server.StandardSaveStrategy.SaveItems (Server.SaveMetrics metrics) [0x00000] 
  at Server.DualSaveStrategy+<>c__CompilerGenerated2.<Save>c__7 () [0x00000] 
  at (wrapper delegate-invoke) System.MulticastDelegate:invoke_void ()
Crash: Backing up...Crash: Backing up...donedonedone

Crash: Generating report...Craffaileds
h: Generating report...Crash: Restarting...failed
Crash: Restarting...donedonedone
There's the crash.

Yes, i've added some scripts and done some minor distro modifications but none of them should make this kind of crash.. :S
Before this crash happens, a player, or some players, gets disconnected from the game, and can't re-connect. When the next save comes, the server crashes.
 

Saumane

Wanderer
I just found out, that
Code:
System.Net.Sockets.SocketException: The socket is not connected
  at System.Net.Sockets.Socket.Shutdown (SocketShutdown how) [0x00000] 
  at Server.Network.NetState.Dispose (Boolean flush) [0x00000]
comes when someone's client crashes. Is there anything i can do to remove this network error?
 
I have the same problem running with mono 1.9.1 on gentoo.
Yes, a few custom scripts, e.g. a welcome gump, use IO-operations like SQL connections.

Anything to look out for?

Or even a solution?
 

Thadeus

Wanderer
I have the same errors when the client crashes or is killed.
I have not, yet, experienced the runuo crash as I am currently the only user.

This is a fresh compile of revision 300 on mono 1.9.1 with literally nothing added other than my own account.
 

Arek

Sorceror
Regarding the socket error:
It looks to me like the problem is that the when a client disconnects uncleanly (i.e. crashes or is killed), the socket is already disconnected when the server finds out about the crash. The server then tries to disconnect the socket, and fails because the socket is already disconnected. The patch attached to this message should fix this by making sure the socket is still connected before trying to disconnect it.

Please note that while I have compiled and run a RunUO 2.0 server with this patch applied on mono 1.9.1, I have not had a chance to connect to the server as of yet. This means that the patch is basically untested, other than the fact that it compiles. Also note that since I haven't tested crashing a client with or without the patch applied, it may not even fix the problem.

If one of you folks wants to be brave and see if this patch fixes the problem for you, then please do. Once I get my server configured a bit better, I'll test it more fully myself. :)

To apply the patch, simply unzip the attached file in the root of your runuo installation and run the following command (as the user you normally use to run RunUO):
patch -p0 < runuo-300-mono-191-netstate.patch​
Then just rebuild runuo.exe and run it as usual.

One last thing to be aware of: This is a core modification, with all the potential problems associated with it. If you choose to use it, you do so at your own risk. I take no responsibility if it completely borks your computer, your shard, or whatever.

-----------------------

Now, on to the Too many open files error (assuming custom scripts are involved in this one):
I don't know myself how to fix this, since I don't know what all scripts and/or core mods are involved, and even then, I may not really be experienced enough with C# or the scripts in question to find or fix the error.

To fix this error you need to go through all of your custom scripts and core modifications and make sure that any files/sockets they use are getting closed properly when the script is done with them, even if an exception occurs while the file/socket is open (use try/catch to deal with this). If you find that your scripts are closing files properly when they're through with them, then the scripts may simply be opening too many files/sockets at once and causing your system to run out of available files/sockets (I think there's a way to increase the number of available sockets on a linux system, but I don't remember what it is offhand).

Hope this helps some folks. :)

--Arek
 

Attachments

  • runuo-300-mono-191-netstate.zip
    411 bytes · Views: 43

ntony

Sorceror
Arek;780080 said:
Regarding the socket error:
It looks to me like the problem is that the when a client disconnects uncleanly (i.e. crashes or is killed), the socket is already disconnected when the server finds out about the crash. The server then tries to disconnect the socket, and fails because the socket is already disconnected. The patch attached to this message should fix this by making sure the socket is still connected before trying to disconnect it.

Please note that while I have compiled and run a RunUO 2.0 server with this patch applied on mono 1.9.1, I have not had a chance to connect to the server as of yet. This means that the patch is basically untested, other than the fact that it compiles. Also note that since I haven't tested crashing a client with or without the patch applied, it may not even fix the problem.

If one of you folks wants to be brave and see if this patch fixes the problem for you, then please do. Once I get my server configured a bit better, I'll test it more fully myself. :)

To apply the patch, simply unzip the attached file in the root of your runuo installation and run the following command (as the user you normally use to run RunUO):
patch -p0 < runuo-300-mono-191-netstate.patch​
Then just rebuild runuo.exe and run it as usual.

One last thing to be aware of: This is a core modification, with all the potential problems associated with it. If you choose to use it, you do so at your own risk. I take no responsibility if it completely borks your computer, your shard, or whatever.

-----------------------

Now, on to the Too many open files error (assuming custom scripts are involved in this one):
I don't know myself how to fix this, since I don't know what all scripts and/or core mods are involved, and even then, I may not really be experienced enough with C# or the scripts in question to find or fix the error.

To fix this error you need to go through all of your custom scripts and core modifications and make sure that any files/sockets they use are getting closed properly when the script is done with them, even if an exception occurs while the file/socket is open (use try/catch to deal with this). If you find that your scripts are closing files properly when they're through with them, then the scripts may simply be opening too many files/sockets at once and causing your system to run out of available files/sockets (I think there's a way to increase the number of available sockets on a linux system, but I don't remember what it is offhand).

Hope this helps some folks. :)

--Arek

Appreciated with your work!

Anyone tried this fix?
 

tolocat

Sorceror
This is not the Runuo question,
is mono bug.

Look here
https://bugzilla.novell.com/show_bug.cgi?id=464201
problem in mono file
Listener.cs
Packets.cs
SocketPool.cs
Socket.cs

Actually Crash question simply not when Dispose() is the m_Socket.Connected condition connected, but is call this NetState.Dispose (bool flush) this movement whether or not wrong. Even if because m_Socket.Connected will not be connected, call m_Socket.Shutdown (SocketShutdown.Both) should as for not cause crash, only will be will cause exception.

look SocketPool.cs
Code:
[FONT=Courier New]public static Socket AcquireSocket()
      {
         lock ( m_FreeSockets )
         {
            if ( m_FreeSockets.Count > 0 )
               return m_FreeSockets.Dequeue();

            ++m_Misses;

            for ( int i = 0; i < m_InitialCapacity; ++i )
               m_FreeSockets.Enqueue( new Socket( AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp ) );

            return m_FreeSockets.Dequeue();
         }
      }

      public static void ReleaseSocket( Socket s )
      {
         /*if ( s == null )
            return;

         s.Close();

         lock ( m_FreeSockets )
            m_FreeSockets.Enqueue( s );*/
      }[/FONT]


Actually when each time aquire new Socket, SocketPool socket which dequeue will be ready to be used from Queue. But I discover in Ubuntu, possesses establishes good Socket in RunUO, even if already Close and Dispose, will not also be released.
This may use: lsof | grep mono | grep socket | wc - l examined.
I try to write very simple C# to try new Close() to fall socket to be able immediately to release, the result will be “will be released”. This question I make me to think that has any place also in reference these Socket, has established the communication socket not to have Flush to fall read/write stream buffer anything, makes him not to be able to release.
Obscure.

Moreover I also found on RunUO Listener.cs discrepence, in this :
Code:
[FONT=Courier New]private void OnAccept( IAsyncResult asyncResult ) {
         Socket listener = (Socket) asyncResult.AsyncState;

         Socket accepted = null; // look here

         try {
            accepted = listener.EndAccept( asyncResult );  // look here
         } catch ( SocketException ex ) {
            NetState.TraceException( ex );
         } catch ( ObjectDisposedException ) {
            return;
         }

         if ( accepted != null ) {  // look here
            if ( VerifySocket( accepted ) ) {
               Enqueue( accepted );
            } else {
               Release( accepted );  // look here
            }
         }

         try {
            listener.BeginAccept( SocketPool.AcquireSocket(), 0, m_OnAccept, listener );
         } catch ( SocketException ex ) {
            NetState.TraceException( ex );
         } catch ( ObjectDisposedException ) {
         }
      }[/FONT]
 
Thanks for that answer, but yet I don't understand why SocketPool.AcquireSocket() is being called without using it's return value. Doesn't this produce just additional unused Sockets?
 

Mark

Knight
Lord_Helmchen;788378 said:
Thanks for that answer, but yet I don't understand why SocketPool.AcquireSocket() is being called without using it's return value. Doesn't this produce just additional unused Sockets?

We use the socket. The socket returned is being passed as an argument to the BeginAccept call.

See: Socket.BeginAccept Method (Socket, Int32, AsyncCallback, Object) (System.Net.Sockets)

SocketPool is a little confusing. It was initially an attempt at reusing sockets. When that didn't work out, we turned it into a block allocator.

You can adjust the number of sockets allocated in each block by adjusting the PooledSockets variable in SocketOptions.cs.

Regardless, this is not the concern of the original poster.

Code:
System.Net.Sockets.SocketException: The socket is not connected
  at System.Net.Sockets.Socket.Shutdown (SocketShutdown how) [0x00000] 
  at Server.Network.NetState.Dispose (Boolean flush) [0x00000]

Errors like the one above are harmless and were simply left in for debugging purposes.

However, if you are seeing something like "System.Net.Sockets.SocketException: Too many open files", then you do have a problem.

That issue is probably due to a leak in Mono (unless you have a lot of clients; if you do, see below). If you aren't running the latest version, try updating.

"In Unix-like systems, file descriptors can refer to files, directories, block or character devices (also called "special files"), sockets, FIFOs (also called named pipes), or unnamed pipes." See: File descriptor - Wikipedia, the free encyclopedia

Your individual user account can run out of file descriptors. Usually each user account is limited to 1024 file descriptors (some system administrators lower this though, are you on a shared server?). Once you realize that each client uses at least one file descriptor (and the server itself uses many for various data files, log files, libraries, etc) it's easy to see how you can run out quickly on larger servers. This can be increased if necessary.

You can check your user limit with the following:
$ ulimit -n
1024

There is also a system wide limit.

You can see your current settings with the following:

$ cat /proc/sys/fs/file-max
52427

# cat /proc/sys/fs/file-nr
3391 969 52427
| | |
| | |
| | maximum open file descriptors
| total free allocated file descriptors
total allocated file descriptors
(the number of file descriptors allocated since boot)
 

Kons.snoK

Sorceror
i think i got same problem..

This exception is fatal, press return to exit
Crash: Generating report...failed
Crash: Backing up...done
at Server.Core.Main (System.String[] args) [0x00000]
at Server.Timer.Slice () [0x00000]
at Server.Misc.StatusPage.OnTick () [0x00000]
at (wrapper remoting-invoke-with-check) System.IO.StreamWriter:.ctor (string)
at System.IO.StreamWriter..ctor (System.String path) [0x00000]
at System.IO.StreamWriter..ctor (System.String path, Boolean append, System.Text.Encoding encoding, Int32 bufferSize) [0x00000]
at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share) [0x00000]
at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean anonymous, FileOptions options) [0x00000]
System.IO.IOException: Too many open files
Error:
at Server.Network.Listener.OnAccept (IAsyncResult asyncResult) [0x00000]
at Server.Network.SocketPool.AcquireSocket () [0x00000]
at System.Net.Sockets.Socket..ctor (AddressFamily family, SocketType type, ProtocolType proto) [0x00000]
System.Net.Sockets.SocketException: Too many open files

i've not understood how to change those values:

$ cat /proc/sys/fs/file-max
52427

# cat /proc/sys/fs/file-nr
3391 969 52427

Edit:

Asking a friend got the solution :)

Edit:
etc/security/limits.conf

Adding:
USERNAME – nofile 10240

Check with ulimit-n
 
Top