Author Topic: roster update during startup (Read 3367 times)

Nytridr · « **on:** May 08, 2008, 06:11:57 pm »

looking at the console it looks like the roster is updating 2x during startup. Not sure if this is just a logging bug or if it is actually happening.

Code: [Select]

rsbot [SETTINGS]        [SAVED] LastRosterUpdate for module members set to 12102
62698 as datatype int
rsbot [ROSTER]  [UPDATE]        Roster update complete. Added 0 members and remo
ved 0 of which 0 was rerolled.
rsbot [GROUP]   [MSG]   [Rising Sun] Rsbot: Roster update completed ::: System r
eady
rsbot [GROUP]   [MSG]   [Rising Sun] Rsbot: Roster update completed ::: System r
eady

I tried to get the first restart on the bot but couldnt.. here is the next restart.. The first one created a few pages of db access errors but this one didnt.

Code: [Select]

rsbot [SETTINGS]        [LOAD]  Loaded settings from database.
rsbot [ROSTER]  [UPDATE]        Starting roster update
rsbot [ROSTER]  [UPDATE]        Roster update ran less than 6 hours ago, skippin
g!
rsbot [CORE]    [INC_GANNOUNCE] Detected org name as: Rising Sun
rsbot [BUDDY]   [LOG]   Zeusdoc logged [on] (MEMBER)
rsbot [BUDDY]   [LOG]   Miharuchan logged [on] (MEMBER)
rsbot [BUDDY]   [LOG]   Jobbii logged [on] (MEMBER)
rsbot [SETTINGS]        [SAVED] LastRosterUpdate for module members set to 12102
62932 as datatype int
rsbot [ROSTER]  [UPDATE]        Roster update complete. Added 0 members and remo
ved 0 of which 0 was rerolled.
rsbot [GROUP]   [MSG]   [Rising Sun] Rsbot: Roster update completed ::: System r
eady

Here is a complete one

Code: [Select]

rsbot [SETTINGS]        [LOAD]  Loaded settings from database.
rsbot [ROSTER]  [UPDATE]        Starting roster update
rsbot [ROSTER]  [UPDATE]        Roster update ran less than 6 hours ago, skippin
g!
rsbot [CORE]    [INC_GANNOUNCE] Detected org name as: Rising Sun
rsbot [BUDDY]   [LOG]   Zeusdoc logged [on] (MEMBER)
rsbot [BUDDY]   [LOG]   Metanyt logged [on] (OWNER)
rsbot [BUDDY]   [LOG]   Archk logged [on] (MEMBER)
rsbot [BUDDY]   [LOG]   Miharuchan logged [on] (MEMBER)
rsbot [BUDDY]   [LOG]   Jobbii logged [on] (MEMBER)

Warning: mysql_connect(): Can't connect to MySQL server on 'localhost' (10048) i
n C:\aobots\bebot test\Sources\MySQL.php on line 122
MySQL error (# 0) on query: Cannot connect to the database server!


Warning: mysql_connect(): Can't connect to MySQL server on 'localhost' (10048) i
n C:\aobots\bebot test\Sources\MySQL.php on line 122
MySQL error (# 0) on query: Cannot connect to the database server!


Warning: mysql_connect(): Can't connect to MySQL server on 'localhost' (10048) i
n C:\aobots\bebot test\Sources\MySQL.php on line 122
MySQL error (# 0) on query: Cannot connect to the database server!


Warning: mysql_connect(): Can't connect to MySQL server on 'localhost' (10048) i
n C:\aobots\bebot test\Sources\MySQL.php on line 122
MySQL error (# 0) on query: Cannot connect to the database server!


Warning: mysql_connect(): Can't connect to MySQL server on 'localhost' (10048) i
n C:\aobots\bebot test\Sources\MySQL.php on line 122
MySQL error (# 0) on query: Cannot connect to the database server!


Warning: mysql_connect(): Can't connect to MySQL server on 'localhost' (10048) i
n C:\aobots\bebot test\Sources\MySQL.php on line 122
MySQL error (# 0) on query: Cannot connect to the database server!


Warning: mysql_connect(): Can't connect to MySQL server on 'localhost' (10048) i
n C:\aobots\bebot test\Sources\MySQL.php on line 122
MySQL error (# 0) on query: Cannot connect to the database server!


Warning: mysql_connect(): Can't connect to MySQL server on 'localhost' (10048) i
n C:\aobots\bebot test\Sources\MySQL.php on line 122
MySQL error (# 0) on query: Cannot connect to the database server!


Warning: mysql_connect(): Can't connect to MySQL server on 'localhost' (10048) i
n C:\aobots\bebot test\Sources\MySQL.php on line 122
MySQL error (# 0) on query: Cannot connect to the database server!

rsbot [SETTINGS]        [SAVED] LastRosterUpdate for module members set to 12102
63828 as datatype int
rsbot [ROSTER]  [UPDATE]        Roster update complete. Added 0 members and remo
ved 0 of which 0 was rerolled.
rsbot [GROUP]   [MSG]   [Rising Sun] Rsbot: Roster update completed ::: System r
eady
^CTerminate batch job (Y/N)? ^C

Blueeagle · « **Reply #1 on:** May 08, 2008, 09:58:35 pm »

Are you by any chance also running a bittorrent client on this machine when this error occures?

Nytridr · « **Reply #2 on:** May 08, 2008, 10:13:13 pm »

no I am not running any kind of bit torrent on the bot computer.

Blueeagle · « **Reply #3 on:** May 08, 2008, 10:23:52 pm »

Is that machine running any other port-intensive programs? Other servers? Is it connectable from the outside world? How is the firewall settings on this machine?

Because the best guess I've come up with is http://dev.mysql.com/doc/refman/5.1/en/can-not-connect-to-server.html

If you look a tad down on the page it indicates high port usage as a possible source of the error.

Could I ask you to try the following:

First: Try using '.' (that is a period) as the host name. That should, as far as I can tell, force the use of a named pipe. It may or may not work as I haven't got a windows installation and I cannot test it myself.

If that doesn't work could I ask you to try the registry modifications that are mentioned in the above web page? That should rule out running out of ports.

Glarawyn · « **Reply #4 on:** May 08, 2008, 10:32:31 pm »

Running netstat -a while that error is going on should confirm or debunk that theory Blue's theory. I'd dump it into a test file for eaiser viewing:

c:\> netstat -a > c:\temp\netstat.txt
netstat -a on my windows box gives me 129 lines...

I'd check to make sure that DEP and the Windows Firewall weren't messing with MySQL. For DEP, right click my computer, properties, advanced, performance, Data Execution Protection. If your are set to Turn on DEP for essential Windows programs and services only you should be fine. If you have it on the all programs setting MySQL might be doing something to trigger DEP.

Also make sure you exclude the MySQL data store files from your AV software's scanning, as well as any other programs that are scanning files on your hard drive. Windows Desktop Search, Google Desktop, Windows Defender. This is just a general best practice for database data store files.

Check your log files, windows error log for messages. Lastly pop open the Windows Services MMC and check to make sure you have MySQL set to restart on any failures, though if you do have a failure requiring a restart of MySQL BeBot will freak out like it is above.

Nytridr · « **Reply #5 on:** May 08, 2008, 11:29:42 pm »

Quote from: Blueeagle on May 08, 2008, 10:23:52 pm

Is that machine running any other port-intensive programs? Other servers? Is it connectable from the outside world? How is the firewall settings on this machine?

Because the best guess I've come up with is http://dev.mysql.com/doc/refman/5.1/en/can-not-connect-to-server.html

If you look a tad down on the page it indicates high port usage as a possible source of the error.

Could I ask you to try the following:

First: Try using '.' (that is a period) as the host name. That should, as far as I can tell, force the use of a named pipe. It may or may not work as I haven't got a windows installation and I cannot test it myself.

If that doesn't work could I ask you to try the registry modifications that are mentioned in the above web page? That should rule out running out of ports.

well doing the registry edits it seems to of done it. I will watch it over the next few days and see how it works out. I think that is what was happening though. The outbound connections that bebot makes during a rosterupdate is huge. but after changing the registry and reboot, I am able to do it with out any errors.

before I was getting alot of

C:\Program Files\MySQL\MySQL Server 5.0\bin\mysqld-nt: Forcing close of thread 395120 user: 'rsbot'

only thing differnt ofc is the threads.

and as far as netstat after the reg fix I get the following. I am not really worried about ports on this computer since it is behind a firewall.. I have 1180 lines during a !rosterupdate just from the bots, there are more ports but that is just from the bots themselves.

Blueeagle · « **Reply #6 on:** May 09, 2008, 01:16:58 am »

Quote from: Nytridr on May 08, 2008, 11:29:42 pm

well doing the registry edits it seems to of done it. I will watch it over the next few days and see how it works out.

Then I think we should call that the fix.

Khalem · « **Reply #7 on:** May 09, 2008, 01:31:06 am »

I have a slight concern that we might actually be overloading default installs which can be considered somewhat a bad thing.

At the same time, I'm not sure there is any way we can reduce the number of queries, although i think we should consider going back to pconnect for mysql if we detect a windows host for multiple reasons.

Nytridr · « **Reply #8 on:** May 09, 2008, 03:26:36 am »

Quote from: Khalem on May 09, 2008, 01:31:06 am

I have a slight concern that we might actually be overloading default installs which can be considered somewhat a bad thing.

At the same time, I'm not sure there is any way we can reduce the number of queries, although i think we should consider going back to pconnect for mysql if we detect a windows host for multiple reasons.

Any possible way to detect the following and adjust accordantly on a windows based machine?

Code: [Select]

#

Locate the following key in the registry:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

#

On the Edit menu, click Add Value, and then add the following registry value:

Value Name: MaxUserPort
Data Type: REG_DWORD
Value: 65534

This sets the number of ephemeral ports available to any user. The valid range is between 5000 and 65534 (decimal). The default value is 0x1388 (5000 decimal).
#

On the Edit menu, click Add Value, and then add the following registry value:

Value Name: TcpTimedWaitDelay
Data Type: REG_DWORD
Value: 30

This sets the number of seconds to hold a TCP port connection in TIME_WAIT state before closing. The valid range is between 0 (zero) and 300 (decimal). The default value is 0x78 (120 decimal).

taken from http://dev.mysql.com/doc/refman/5.1/en/can-not-connect-to-server.html

Not sure if this would be to much or not. just an idea

Blueeagle · « **Reply #9 on:** May 09, 2008, 07:24:59 am »

Quote from: Khalem on May 09, 2008, 01:31:06 am

I have a slight concern that we might actually be overloading default installs which can be considered somewhat a bad thing.

At the same time, I'm not sure there is any way we can reduce the number of queries, although i think we should consider going back to pconnect for mysql if we detect a windows host for multiple reasons.

I for one do not have any issue with making these alterations to a windows installation. These were the defaults prior to win2003 and the reason they were altered, as I have been told, was to mitigate the effects of the mssql exploit. Not that this does not in any way attempt to patch the said exploit it just doesn't hang the system totally when exploited which in my opinion is a bad thing. A hung system needs to be looked into much faster and the exploit patched much sooner than a system that's merely slugish.

Alreadythere · « **Reply #10 on:** May 09, 2008, 09:36:07 am »

For some reason we were using mysql_connect() with new_link=true. Which is bad when doing lot's of queries in a short time.

Changed in 0.4 and trunk.

Blueeagle · « **Reply #11 on:** May 09, 2008, 10:20:05 am »

Khalem: Care to enlighten us on "- Properly set new_link flag when creating the connection." ? :p

Khalem · « **Reply #12 on:** May 09, 2008, 03:44:00 pm »

Errr... oops?

We also need to remove the explicit mysql_close calls for the change to have any effect as those are only needed with pconnects afaik.

I won't have time to look at it until after the weekend as I'm not at home.

As for the bot changing TCP/IP settings in the windows registry, that sounds like something we definately never want to do. One thing is to detect and alert, another entirely to detect and mess with system settings of peoples computers/servers.

Alreadythere · « **Reply #13 on:** May 09, 2008, 04:00:58 pm »

mysql_close() is never called anyways.

Temar · « **Reply #14 on:** May 09, 2008, 11:41:25 pm »

Fixed the 2x Roster update
the Problem was Cron was running on connect instead of 30secs later(default)

and there is a hourly cron check that notify and buddy list is correct
howeva buddylist is empty at connect as it hasnt loaded them yet
so it does a roster update in an attemp to solve that

Author Topic: roster update during startup (Read 3367 times)

Nytridr

roster update during startup

Blueeagle

Re: roster update during startup

Nytridr

Re: roster update during startup

Blueeagle

Re: roster update during startup

Glarawyn

Re: roster update during startup

Nytridr

Re: roster update during startup

Blueeagle

Re: roster update during startup

Khalem

Re: roster update during startup

Nytridr

Re: roster update during startup

Blueeagle

Re: roster update during startup

Alreadythere

Re: roster update during startup

Blueeagle

Re: roster update during startup

Khalem

Re: roster update during startup

Alreadythere

Re: roster update during startup

Temar

Re: roster update during startup