collapse collapse
* User Info
 
 
Welcome, Guest. Please login or register.
* Search

* Board Stats
  • stats Total Members: 989
  • stats Total Posts: 18366
  • stats Total Topics: 2501
  • stats Total Categories: 7
  • stats Total Boards: 35
  • stats Most Online: 1144

Author Topic: Persistent Whois cache with out-of-bot upgrading  (Read 85095 times)

0 Members and 2 Guests are viewing this topic.

Offline nebhyper

  • BeBot User
  • **
  • Posts: 62
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #135 on: January 10, 2007, 05:46:38 pm »
wow, for 40k entries in my DB takes about 27mins to run. 

638min runtime... just wow!  I think max I have had is 35 mins for 40k entries!
Siocuffin (Squad Commander of United Notum Federation)
alts: Nebhyper, Nebalmighty.

Offline Malosar

  • BeBot Expert
  • ****
  • Posts: 259
  • Karma: +0/-0
    • http://www.lowerdimension.com
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #136 on: January 10, 2007, 05:56:04 pm »
Well I have my delay set at 1000ms instead of 100ms, but thats a lot longer than usual.

Normal day:

26min 31sec runtime for org updates!
75min 24sec total runtime!
Eternalist
General of The Syndicate

Offline Alreadythere

  • BeBot Maintainer
  • BeBot Hero
  • ******
  • Posts: 1288
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #137 on: January 10, 2007, 07:00:11 pm »
Hm, I got like 80mins for org and 440mins total update time usually, with 110k total entries, 80k of which are orged.

Offline Alreadythere

  • BeBot Maintainer
  • BeBot Hero
  • ******
  • Posts: 1288
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #138 on: January 16, 2007, 12:44:01 pm »
Ported the 0.2.10 get_site function to whois-update.php.
Now the script doesn't produce endless warnings due to timeouts on the http queries anymore.

Just replace the old script with the new version, linked in the first post.

Offline bmoscato

  • BeBot User
  • **
  • Posts: 51
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #139 on: January 23, 2007, 02:28:25 am »
I'm still having some issues running the w_update.php.

I configured the whois-update.conf and have it in the same directory as my w_update.php file.  When I run the php file I get:


Fatal error: Call to undefined function mysql_connect() in C:\BeBot\w_update.php
 on line 117

----------

Line 117: $link = mysql_connect("localhost", $username, $password)

----------

select COUNT(*) from AOBot' at line 1
mysql> select COUNT(*) from whois;
+----------+
| COUNT(*) |
+----------+
|        2 |
+----------+
1 row in set (0.00 sec)

Any thoughts?

Bryan

Offline bmoscato

  • BeBot User
  • **
  • Posts: 51
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #140 on: January 23, 2007, 04:41:21 am »
I made a change to my php.ini file that included:

extension = php_mysql.dll
extension_dir = ./php_extension/

When the script is run the following is returned:

'extension' is not recognized as an internal or external command,
operable program or batch file.


================================================
2 http queries for org information done!
2 total http queries done!
295 entries in database associated with an org modified!
295 entries in database modified!
295 entries in the whois cache!
0min 7sec runtime for org updates!
0min 7sec total runtime!
================================================

Am I doing something wrong?  After reading through this post it appears that most people have over 40k records returned and I'm only returning 295.

Thanks in advance.

Bryan

Offline jjones666

  • Contributor
  • *******
  • Posts: 353
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #141 on: January 23, 2007, 08:32:50 am »
It depends on what modules you are running.  If you just run the updated relay_guild, this will check only the cached data when someone joins the privategroup.

The way the whois cache works is that whenever it is queried, it will add an entry if it isn't found (downloaded the old way from FC site).  When you manually update the cache via that script instead of updating each single person, it will download the entire org data for each org it sees and add each person from that org.

So to fully populate it, you should update a few other plugins to take advantage of it.  You can look here for a mostly complete list of available plugins.  Ones that (imo) you should update would be the whois module and the towerattack module as these take great advantage of the caching features.  Once you have installed the whois update, then !whois one person from each org you can think of and run your update script again, it will soon fill up to a large level :-)  You should also notice a nice speed boost in the bot whenever the whois cache is utilised (especially with !whois requests as previously this could take a long time if the FC site is slow).

Let me know if you need any further information.

Cheers,

-jj-
« Last Edit: January 23, 2007, 08:41:01 am by jjones666 »

Offline Alreadythere

  • BeBot Maintainer
  • BeBot Hero
  • ******
  • Posts: 1288
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #142 on: January 23, 2007, 08:35:58 am »
Not sure what's up with the extension for you. It could be though that you don't need it - that's the case if mysql is compiled into your php.exe.

The low number of entries results most likely of just a few entries in your cache. For the exact description read jjones666's post, he was a few mins faster then me :)

Offline jjones666

  • Contributor
  • *******
  • Posts: 353
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #143 on: January 23, 2007, 08:39:54 am »
Btw, on the php.ini issue, here is mine:

Code: [Select]
; Error handling and logging ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
display_errors = On
display_startup_errors = On
log_errors = On
log_errors_max_len = 0
error_log = ./log/php_errors.txt
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

; BeBot will normally use about 10-13megs with default
; modules. Default php memory limit is 8megs.
; Raise default on hosts that permitt it.
memory_limit = 20M

; Mainly for windows where additional modules are always required
extension_dir = ./php_extension/

I'm not sure of any need to put that extension line in if your whois_update.php and whois_update.conf files are in the root directory of your bot.  The error handling on the first few lines isn't essential, just useful.

Edit:
I'm assuming your directory structure is bebot standard and you do have a directory called php_extension in the root (which contains the needed php_mysql.dll file along with some others).

-jj-
« Last Edit: January 23, 2007, 08:43:19 am by jjones666 »

Offline bmoscato

  • BeBot User
  • **
  • Posts: 51
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #144 on: January 23, 2007, 09:59:54 pm »
Hey Guys,

Thanks for the responses.  The extension error was because I had a brain fart last night and added the @extension = php_mysql.dll to my batch file.

But now I have another question regarding caching TowerAttack and Whois.  I’ve placed the updated php files in my bots modules directory but I’m not sure what to do to force the TowerAttack module to update.

Also, after restarting the bot the log returns these errors:

Code: [Select]
MySQL error (# 0) on query: SELECT nickname, level, rank_name, profession, id, a
ilevel FROM members WHERE id = 590021014
Unknown column 'ailevel' in 'field list'
MySQL error (# 1) on query: UPDATE members SET lastseen = 1169584767 WHERE nickn
ame = 'Cero'
Unknown column 'lastseen' in 'field list'
MySQL error (# 2) on query: SELECT nickname, level, rank_name, profession, id, a
ilevel FROM members WHERE id = 925463630
Unknown column 'ailevel' in 'field list'
MySQL error (# 3) on query: UPDATE members SET lastseen = 1169584767 WHERE nickn
ame = 'Goldfist'
Unknown column 'lastseen' in 'field list'

MySQL error (# 4) on query: SELECT nickname, level, rank_name, profession, id, a
ilevel FROM members WHERE id = 1575319688
Unknown column 'ailevel' in 'field list'
MySQL error (# 5) on query: UPDATE members SET lastseen = 1169584769 WHERE nickn
ame = 'Alphaheals'
Unknown column 'lastseen' in 'field list'
MySQL error (# 0) on query: SELECT id, rank, level, ailevel, lastseen, updated F
ROM members WHERE id = 814199987
Unknown column 'ailevel' in 'field list'
MySQL error (# 1) on query: REPLACE INTO members (id, nickname, firstname, lastn
ame, rank, rank_name, level, profession, gender, breed, ailevel, aititle, lastse
en, updated)
                      VALUES ('814199987',
                              'Bloodstain', '',
                              '', '0',
                              'President', '216',
                              'Engineer', 'Male',
                              'Nano', '15',
                              'Competent', '0', '1169584814')

Does anyone know what would cause this?

Regards,

Bryan

Offline Alreadythere

  • BeBot Maintainer
  • BeBot Hero
  • ******
  • Posts: 1288
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #145 on: January 23, 2007, 10:37:07 pm »
I'd guess you are using some updated Members module, but forgot to add the new fields to your members table.

Offline Malosar

  • BeBot Expert
  • ****
  • Posts: 259
  • Karma: +0/-0
    • http://www.lowerdimension.com
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #146 on: January 23, 2007, 10:49:55 pm »
Or your using new towerattack and whois components but haven't updated your roster and thus don't have the required fields.
Eternalist
General of The Syndicate

Offline bmoscato

  • BeBot User
  • **
  • Posts: 51
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #147 on: January 23, 2007, 11:08:03 pm »
Hi,

I am using the new using the new towerattack and whois module and I did update the roster so I’m not sure what’s wrong now.  I don’t get these errors with the default php files.  If I delete the members table will it be automatically recreated?

Bryan

Offline Xenixa

  • Contributor
  • *******
  • Posts: 307
  • Karma: +0/-0
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #148 on: January 24, 2007, 12:22:53 am »
Roster_GUILD.php will rebuild and populate the members table if members table is deleted.
<<< Hack's in Zend Studio

All my Custom Bebot files may be Found Here <-clicky

Offline Nytridr

  • BeBot Expert
  • ****
  • Posts: 262
  • Karma: +0/-0
    • Rising Sun
Re: Persistent Whois cache with out-of-bot upgrading
« Reply #149 on: February 26, 2007, 03:19:21 am »

I configured the whois-update.conf and have it in the same directory as my w_update.php file.  When I run the php file I get:


Fatal error: Call to undefined function mysql_connect() in C:\BeBot\w_update.php
 on line 117

----------

Line 117: $link = mysql_connect("localhost", $username, $password)
[/quote]

umm having the same problem here.. What did you find out?


Edit:
Looks like it dont like php5...used an older version.. one that came with .2 bot.. and it kind of works..

Code: [Select]
Warning: mysql_escape_string(): This function is deprecated; use mysql_real_esca
pe_string() instead. in C:\aobots\rsbot\aofbot\whoisupdate\whois-update.php on l
ine 224

Warning: mysql_escape_string(): This function is deprecated; use mysql_real_esca
pe_string() instead. in C:\aobots\rsbot\aofbot\whoisupdate\whois-update.php on l
ine 234


================================================
3 http queries for org information done!
4 total http queries done!
1265 entries in database associated with an org modified!
1266 entries in database modified!
1266 entries in the whois cache!
0min 37sec runtime for org updates!
0min 38sec total runtime!
================================================
« Last Edit: February 26, 2007, 03:43:21 am by Nytridr »
Co-Prez of Rising Sun RK1 (1st & only org I will ever belong to)

 

* Recent Posts
Com bot module by bitnykk
[November 25, 2024, 05:36:11 pm ]


0.8.x updates for AO by bitnykk
[June 23, 2024, 03:19:47 pm ]


0.8.x updates for AoC by bitnykk
[June 23, 2024, 03:19:44 pm ]


[AoC] special char for items module by bitnykk
[February 09, 2024, 09:41:18 pm ]


BeBot still alive & kicking ! by bitnykk
[December 17, 2023, 12:58:44 am ]

* Who's Online
  • Dot Guests: 339
  • Dot Hidden: 0
  • Dot Users: 0

There aren't any users online.
* Forum Staff
bitnykk admin bitnykk
Administrator
Khalem admin Khalem
Administrator
WeZoN gmod WeZoN
Global Moderator
SimplePortal 2.3.7 © 2008-2024, SimplePortal