BeBot - An Anarchy Online and Age Of Conan chat automaton
Archive => Anarchy Online Archive => 0.2.x Custom/Unofficial Modules => Topic started by: Alreadythere on December 02, 2005, 12:29:02 pm
-
As posted here (http://bebot.link/index.php/topic,220.0.html) I am using a cache for all whois entries. Read the linked thread for more information, posting just the files here.
I'm using one bot module, and an out-of-bot cronjob for this (running on linux, but there are ways to automatically run jobs at specific times under most or all OS supported by bebot).
The bebot module for bebot branch 0.2.x (http://www.m8y.de/ao/bebot/whois/WhoisCache.phps)
The script to update the cache (http://www.m8y.de/ao/bebot/whois/whois-update.phps) make sure this script is not in the modules or core directory!
Save as .php
The config file for the update script, add the infos here (http://www.m8y.de/ao/bebot/whois/whois-update.conf) has to be in the same directory as the update script
Notes:- To use the the cache you need to modify all get_site() calls done to get user information. Just repace the get_site() with $this -> [bot ->] lookup($username), and modify the result handling. The return value of the lookup() function has speaking names, so it shouldn't be hard.
- Don't run the script while FC updates it's database, you won't get results then. The update is usually somewhere between 09:00 GMT and 11:00 GMT, I am running the script at 13:00 GMT, once daily, as the DB isn't updated more the once daily either.
Updated version (12/14/2005):- removing stale entries after user-defined time now. Done in the update script, default is set to 36h.
- checks for stale entries in the lookup function, after user-set time (default 26h) it automatically queries FC's site for an update, and writes that into the table.
Updated version (12/16/2005):- names of existing orgs get updated now
Updated version (04/17/2006):- smarter script
- script divided in config file and update part for easier updating in the future
Updated version (10/03/2006):- set_time_limit(0); set, as long as save_mode isn't enabled the script won't be aborted due to too long execution time
- update-script supports used defined addons after all updates are done, use the file whois-update.addon.
- mysql_escape_string() and stripslashes() added to remove errors with strange characters in names. First name, last name and org names are escaped.
- Added some more information to the output when the update script is done
- Modified the cache to use the same field names and order of fields in the table as the version shipped with the 0.3 branch. This is a purely database change, the names in the returned array haven't been changed. So no further modification to bebot files are needed besides replacing the WhoisCache.php. To avoid wrong behaviour of the cache you need to update the table! The SQL code below will do the actual updating, it's in a comment in the WhoisCache.phps file too. Replace whois with the name of your table. Depending on the size of the cache those update commands may take a few minutes to run.
ALTER IGNORE TABLE whois ADD id int(15) NOT NULL default '0' FIRST ;
ALTER IGNORE TABLE whois CHANGE name nickname VARCHAR(20) NOT NULL;
ALTER IGNORE TABLE whois ADD pictureurl VARCHAR(100) NOT NULL AFTER org_rank_id, ADD used BIGINT(25) NOT NULL AFTER pictureurl;
ALTER IGNORE TABLE whois ADD INDEX (used);
ALTER IGNORE TABLE whois ADD INDEX (updated);
Updated version (10/05/2006):- worked around some errors
- removed all calls to custom function (tablenames, settings). Cache can now be added without any modifications to Bot.php
Updated version (01/16/2007):- update script handles timeouts on http queries now
Updated version (03/30/2007):- Added fix for the notices in the xmlparse() and get_site() function
- updating pictureurl now too, no idea why I forgot that
- optimized handling of org members a bit
-
Here as example for the changes my current whois_player() function in Whois.php.
If you compare it to the default one in bebot, I just replaced the get_site() call and the further parsing with the lookup() call.
/*
Returns info about the player
*/
function whois_player($name)
{
$who = $this -> bot -> whois -> lookup($name);
if (!empty($who["nick"]))
{
$at = "<font color=#00ff60>(AT " . $who["at"] . " - " . $who["at_name"] . ")</font> ";
$result = "\"<font color=#ffff00>" . $who["nick"] . "</font>\"";
if (!empty($who["firstname"]) && ($who["firstname"] != "Unknown"))
$result = $who["firstname"] . " " . $result;
if (!empty($who["lastname"]) && ($who["lastname"] != "Unknown"))
$result .= " " . $who["lastname"];
$result .= " is a level <font color=#ffffff>" . $who["level"] . "</font> " . $at . $who["gender"] . " " . $who["breed"] . " <font col
or=#ffffff>" . $who["profession"] . "</font>, " . $who["faction"];
if (!empty($who["rank"]))
$result .= ", " . $who["rank"] . " of " . $who["org"] . ".";
else
$result .= ".";
}
else
$result = "www.anarchy-online.com was too slow to respond.";
return $result;
}
-
in your update script, i had to change one line:
$sqlquery = "SELECT * FROM " . $tablename . " WHERE updated <= " . $thistime;
for some reason you'd used single-quotes before the SELECT and after the <=.
gave an error that way :)
worked like a charm otherwise, though. thanks a ton! :)
-
Oh my, yes, you are right there.
I had a fixed tablename in my script (didn't plan on post it at first), and typical for me I didn't test the script I posted. Should start to do that :)
Another note: you will get some warnings as output if the script tries to update a character that is deleted. Haven't done anything to remove those toons.
-
I like this very much. However i have one request.
Using an external script to update the cache is nice, but i do feel there should be a fallback option for those who do not wish to run an external script.
Add a simple check to the whois that checks the last updated time, if it has not been updated in over 30-36 hours, update the cache for that person.
I still think a file based cache could be better suited though, but this is a step up from the current system regardless.
-
Add a simple check to the whois that checks the last updated time, if it has not been updated in over 30-36 hours, update the cache for that person.
I'll try to go around to it, I already got the updated field, so it won't be that hard really (just one check more).
I still think a file based cache could be better suited though, but this is a step up from the current system regardless.
Personally I prefer database systems, as it's hard for any filesystem or self-written function to reach the speed a professional database has in lookups.
Besides, even with 38k entries now in my table the total size is only 4.5mb. Easy to handle memory wise.
-
Been playing with this one today. I like it alot. :)
I don't mind large db tables as I have enough machine to handle it. What I like most about it is I don't have alot of members using up my band width doing whois againist FC's database.
Doing whois would often times just time out if I had alot of members on chatting away and issuing other commands and what not.
BTW since I run on Win2K (and this works for XP also) if you want to schedule this to run once a day here's a handy document to look at.
http://windows.about.com/od/customizationsforwindows/l/aa001022b.htm
I created a batch file for the Windows Task Scheduler to run, seems to be the easiest method.
w_update.bat@cls
@php w_update.phpPlace that .bat in same Folder as php.exe and the Update php file. Which I renamed w_update.php since I have other update scripts I been playing with.
-
i have no need for anything beyond what is already done. i may cook up a script to remove deleted characters at some point, but really it's "good enough" for my purposes.
Thanks again, Already ;)
/Aka
-
Modified the Guest Join announcement function I had stuck in Relay_GUILD.php so players joining guest channel don't keep getting looked up on FC's site.
Just insert this at the bottom if your using Already's Whois cache:<?
/*
Sends info to private group and guild chat on guest who joined
*/
function pgjoin($name)
{
$highlight = $this -> highlight;
// Gets character info from whois cache or anarchy online website
$who = $this -> bot -> whois -> lookup($name);
if (!empty($who["nick"]))
{
$at = "<font color=#33FF33>(AL " . $who["at"] . " - " . $who["at_name"] . ")</font> ";
$result = $highlight . "\"" . $who["nick"] . "</font>\"";
if (!empty($who["firstname"]) && ($who["firstname"] != "Unknown"))
$result = $who["firstname"] . " " . $result;
if (!empty($who["lastname"]) && ($who["lastname"] != "Unknown"))
$result .= " " . $who["lastname"];
$result .= " a level " . $highlight . $who["level"] . "</font> " . $at . $who["gender"] . " " . $who["breed"] . " " . $highlight . $who["profession"] . "</font>, " . $who["faction"];
if (!empty($who["rank"]))
$result .= ", " . $who["rank"] . " of " . $who["org"] . " joined <botname>.";
else
$result .= ", joined <botname>.";
}
$this -> bot -> send_pgroup($result);
$this -> bot -> send_gc($result);
}
?>Note: Also using the highlight color in this one so function Relay needs to look like this in the same module:function Relay (&$bot)
{
$this -> bot = &$bot;
if(!$this -> bot -> highlight_color) {
$this -> highlight = "<font color=#FFFF00>";
} else {
$this -> highlight = "<font color=" . $this -> bot -> highlight_color . ">";
}
}And just one other tid bit. If you want to announce people Parting the bots private channel do this to Online.php -> Look for:/*
This gets called if someone leaves the privgroup
*/
function pgleave($name)
{
if (isset($this -> pgroup[$name]))
unset($this -> pgroup[$name]);
}Change to:/*
This gets called if someone leaves the privgroup
*/
function pgleave($name)
{
if (isset($this -> pgroup[$name]))
unset($this -> pgroup[$name]);
$this -> bot -> send_gc($name ." has Left the Guest Channel");
}
-
this should come standard with version 3 :) with an on/off switch of course or something like that
it works like a charm btw ^^
its almost spooky
-
mysql> select COUNT(*) from whois;
+----------+
| COUNT(*) |
+----------+
| 41142 |
+----------+
yes.. very spooky ;)
the only change i am considering is one that will let me put it in a database called 'whois' so multiple bots can access it more easily..
-
49291 entries, growth definitly slowed down for me.
the only change i am considering is one that will let me put it in a database called 'whois' so multiple bots can access it more easily..
That's what I'm doing, all my 8 bots access my whois-table.
I've done it using my get_tablenames function, so I can easily share tables across bots. I posted a version of it in the 0.3.0 thread I think.
-
Well, ummm.. bummer, the offline Character database at FC is now behind a Squid.
Kinda saw this coming. Now if too many requests are made on the those XML files in a short period of time, you'll start to get connection refused messages from the Squid server... bleh
Ex:
While trying to retrieve the URL: http://216.74.158.92:8080/content/community/people/lookup/chars.html
The following error was encountered:
* Connection Failed
The system returned:
(111) Connection refused
The remote host or network may be down. Please try the request again.
Generated Wed, 14 Dec 2005 13:08:36 GMT by www.worldofmidgard.com (Squid/2.2.STABLE5)
-
Hm, could it be that they took down their database during the downtime? Haven't checked my logs yet though, not sure what the cache did today...
-
Updated version:
- removing stale entries after user-defined time now. Done in the update script, default is set to 36h.
- checks for stale entries in the lookup function, after user-set time (default 26h) it automatically queries FC's site for an update, and writes that into the table.
Updated files in the links in first post.
-
Tested my script after your report Xenica, didn't have any error messeges besides the ones about non-existing HTML sites. And only a few there either, still think those are no longer existing characters.
-
Thinking now that error occured as they were updating their network stuff.
The AO website is behind a Squid now it seems. As long as there squid isn't to slow at caching we should be ok :)
-
Updated the script, forgot to check for renames of orgs. Org names get updated to new ones now, only change.
-
Hmm I've suddenly started to get
"b>Warning</b>: fopen(http://www.anarchy-online.com/character/bio/d/2/name/noahsdove/bio.xml): failed to open stream: HTTP request failed! HTTP/1.0 503 Service Unavailable
When using this, anyone else has?
-
I get that also .. repeatedly for a group of names. I think the FC servers can't keep up with the requests.
Which reminds me. Been meaning to ask. Why pull individual character Bio's if they belong to an org? Should pull the Org XML file and parse that for the character data. Would be less http requests that way. You would need to keep a routine to handle non-org'd characters sure, but for the script update I think the Org XML file should be used. Just my 2 cents.
-
Hmm I've suddenly started to get
"b>Warning</b>: fopen(http://www.anarchy-online.com/character/bio/d/2/name/noahsdove/bio.xml): failed to open stream: HTTP request failed! HTTP/1.0 503 Service Unavailable
When using this, anyone else has?
Hm, same in my logs. Strange thing is - if I query the pages manually I get a correct reply.
Could be that the speed is too high, so some DOS protection stops the further queries. If that's the case, a small delay between each HTTP query could help. But just guessing here, don't have a real clue.
Which reminds me. Been meaning to ask. Why pull individual character Bio's if they belong to an org? Should pull the Org XML file and parse that for the character data. Would be less http requests that way. You would need to keep a routine to handle non-org'd characters sure, but for the script update I think the Org XML file should be used. Just my 2 cents.
Check my update script again please ;)
It gets a list of all org IDs in the cache first, then updates the entries for all members of those orgs. And only after that it queries the bios for all not yet updated characters.
-
I would like a version that stores the cache in CVS, cause I dont have the db on the same computer as the bot.
-
I would like a version that stores the cache in CVS, cause I dont have the db on the same computer as the bot.
CVS isn't a database, so putting the cache there doesn't make any sense. Actually I think the overhead would be steep.
And the database and the update script don't need to be on the same computer, just enter the correct server information in the script. It should work over network too (never tested, but don't see any reason why it shouldn't work).
The variable to set is $dbserver - which I'm defaulting to localhost.
-
Current Character Count in Whois Cache: 62495 :o
Heh ever since I converted the modified Towers module by Crazied to use the Whois functions of this module that table has been growing like mad fast. ;D
-
Hehe, yes, got a similar number in my cache.
Growth slowed down a bit though :)
On a side note, currently contemplating to add UIDs to the cache. Would allow an easier correlation with a shrunken members table (see that thread), and could help in some cases with reverse ID lookups. As it's extremely rare that there is a change in the ID/name combo. Still possible on renames.
-
I must be missing something obvious, but I'm getting this error when running the update script from command line on winXP (same box that has the mysql database and the bot running on it)
C:\BeBot>php whois_update.php
Fatal error: Call to undefined function mysql_connect() in C:\BeBot\whois_update
.php on line 30
Line 30 is this block:
$link = mysql_connect("localhost", $username, $password)
or die("Could not connect : " . mysql_error());
mysql_select_db($dbname) or die("Could not select database");
What am I missing here?
-
Most likely a correct php ini, which has to include the module path for the mysql extension.
Something like
extension=php_mysql.dll
extension_dir = "./"with extension_dir adapted to your environment. (Not quite certain about the correct extension name, the windows version I got has mysql included).
-
Thanks, that fixed it.
-
Hm, i cant get this to work...
Well I've added all at bot class,
d/led files and set em up..
I get a big "nothing"
C:\BeBot_v0.2.3>php w_update.php
C:\BeBot_v0.2.3>
more info.. I got Whoiscache.php in /modules, I got w_update.php in bebot directory.. I got bot class updated in bot.php.. cant think of what i wrong, any help appreciated.
Also modified Whois.php to use this already there module.. but Dont want to update ;)
-
Did you add the needed configuration information in the beginning of the update file?
It doesn't read the conf/Bot.conf nor conf/Mysql.conf.
Only thing I can think of.
Anyways, I have updated my script a bit, it doesn't try to add empty names anymore and gives some more output.
And it's split in a config file and a php script now, for easier updating in the future.
The script, rename to php (http://www.m8y.de/ao/bebot/whois/whois-update.phps)
The config file, add the infos here (http://www.m8y.de/ao/bebot/whois/whois-update.conf)
-
hm atleast now works, and yes cofigured all.
But now i get this :
C:\bebot3.2\whoisbebot>php whois-update.php
================================================
0 http queries done!
0 entries in database modified!
0 entries in the whois cache!
0min 0sec total runtime!
================================================
C:\bebot3.2\whoisbebot>
Probly FC update Database i think:P
-
Alreadythere can you modifay yor script to update members table?
-
Probly FC update Database i think:P
Unlikely, as FC updates their server between 8-10 GMT usually.
You have to add entries to the table first, otherwise the script doesn't have anything to update. It does not go and pull a full mirroring of the FC databse.
Modify whois to use the cache, query a few people in different orgs, and run the script again - it should pull infos then.
-
Alreadythere can you modifay yor script to update members table?
It's not supposed to do that.
And it won't be possible anyways, as the UID is needed in the members table. And there is no way to get that from outside AO sadly. So you will always need a bot to update your members table.
-
but got truble with update members.....
and one way its out side ubdate in task command other its plugin ro make this
i teast all rosters plugin wit out any update im member info.....
can you help?
-
I made roster_guild.php check the whoiscache: $result = $this->bot->db->select("SELECT * from whois where org_id='".$this->bot->guildid."'");
If it isn't empty(), I loop it, otherwise (some error with the db or whatnot) I use the old method. I've modified my roster way too much, else I'd post the file here.
-
You can get whois XML data from Auno as well, so it's possible to query whois cache, anarchy-online.com, and then auno if AO.com is down...
For Auno, only the URL changes...
http://auno.org/ao/char.php?output=xml&dimension=1&name=Auno
-
If you go an use the cache for member list updates, you should add an index over the org ids (or org names).
Otherwise searching in the table can take quite a bunch of time.
-
Im currently adding the whois cache with some modifications to the BeBot 0.3 branch.
It will function fine standalone without out of bot updating, but can also be combined with out of bot updating :)
However, since i use character id's as the ID and for internal purposes, people cannot be added to the whois cache from outside the bot.
Im also doing a complete overhaul of get_site which will require minor changes to handle new return format just as a heads up.
The change in return format is to easier accomodate proper error messages, something which have been bugging me for some time.
The code should make it into the trunk sometime tomorrow if not earlier.
-
I got an updated version of the whois module here (http://www.m8y.de/ao/bebot/whois/WhoisCache.phps.IDs).
It does include IDs (added them, haven't gotten around to use them yet though), and is a bit smarter then the last version I posted concerning which information to use and add.
The ID problem could be solved by adding a get_uid call each time a query is done which doesn't return an ID. Wouldn't be much overhead, and still work without any changes with the outside updating.
This version does work without outside updates too, would need a lower setting for whois_cache_persist_time though.
-
The fallback to database version is a nice one wich i didn't think about.
I just commited the initial version of the whois cache. I'll add the fallback later :)
[Edit]
I should point out that i got slightly annoyed with the inconsistencies of some of the whois array names used in various places in BeBot and proceeded to alter a few of them, including the ones now given by the whois cache to make everything more consistent.
-
I should point out that i got slightly annoyed with the inconsistencies of some of the whois array names used in various places in BeBot and proceeded to alter a few of them, including the ones now given by the whois cache to make everything more consistent.
Hehe, yes, too many people doing their own stuff.
I think I took the array out of the original whois function.
Not that it really matters, calls just need to be adapted once for consistency.
-
Just one question - why didn't you keep whois in it's own module?
Modules can be kept up-to-date easier, and the way I see it, the core modules are modules the bot depends on anyways.
-
Main reason is that i consider the actual function of the whois lookup to be a completely integral part of the bot equal to the other core functions in Bot.php.
As so many modules depend on it, it made sense that it be a bot -> function.
I agree though on the easier to update argument, especially if you want to use a customized version of it. On the other hand though the actual whois lookup should rarely ever change except when Funcom changes XML aside from functional enhancements like smartening it up a bit.
Im open to feedback of course on any counts :)
-
Yeah, I don't expect that function to get changed either tbh.
I'm just no friend of needless bloating up of the Bot class. Even though I've done it with my settings functions too. But going to post a core/Settings module for those functions soon.
Main reason I posted it as module was the either upgrading in the old bot.
-
Speaking of a settings module. I was thinking along the same lines. I couldn't help but notice I kept having to make changes to different versions of bot.php in reguards to get and set settings. Started to work on a core/ module that would handle that stuff including the table name stuff for multi-bot databases.
Haven't had alot of time to work on it though as of late but it is something I'd like to see 'officialized' with the bot. Whois in the Bot.php makes sense to me also as thats the original place for it even though I'm using Already's stand alone version.
BTW, Core modules are treated as part of the Bot Class when they're loaded aren't they? Just have to add the new core class var to the Bot class when you drop one into the core/ folder.
-
BTW, Core modules are treated as part of the Bot Class when they're loaded aren't they? Just have to add the new core class var to the Bot class when you drop one into the core/ folder.
Yes and no.
Yes for all modules.
In core modules you have to be a bit more carefull. You should not call functions out of other core modules outside the class declaration (where all the create table calls are atm), nor in the constructor. As you cannot predict the orders the modules are loaded in you cannot guarantee that the functions are available at the needed places. After the contruction phase you don't have this problem anymore.
-
great module :D
but from what i understand, it takes several number of ppl from FC db and put it in the whois table, right?
if so.. how does it know when someone that already in the whois table lvled up or rerolled?
oh, and why it takes 6 minutes+ to load the bot every time? :/
Edit:
now the bot not loading at all
giving me this errors
Warning: fopen(http://www.anarchy-online.com/character/bio/d/1/name/wergggg/bio.xml): failed to open stream: HTTP request failed! HTTP/1.0 503 Service Unavailable
in C:\red\modules\whois-update.php on line 7
Warning: fgets(): supplied argument is not a valid stream resource in C:\red\modules\whois-update.php on line 10
-
Guess you started the bot during FCs site update - it will time out then if it has to lookup the FC site.
Rerolls will be detected as soon as FC updates the entry, it will just be replaced then.
-
Umm just a quick note - make sure the php-update file isn't in the bebot modules directory - it should not be loaded at bot startup but rather run manually every day or so. This could explain why the bot takes so long to startup (I think my update script can take up to an hour to process fully). I put my update script in the bebot root directory.
This module has been replaced by a built in who function in the new version of Bebot (unreleased) and I believe that automatically updates itself.
Cheers,
-jj-
-
nah Already, it wasnt during fc db update
but what happended is what jj said :D
thanks =)
will move it from there :)
should i keep the whoiscache.php in module dir?
-
This module has been replaced by a built in who function in the new version of Bebot (unreleased) and I believe that automatically updates itself.
Only partly true.
Both versions will try to update an entry after some time without update.
But only the script does mass updates, the other way only updates entries when they are queried.
-
should i keep the whoiscache.php in module dir?
I've got mine in core/, but module/ should work too. It has to be in one of those two though.
-
nah Already, it wasnt during fc db update
but what happended is what jj said :D
Added a note in my first post about this.
-
Do you have to create tables in the database or does it do it automatically because mines asking for some tables ???
-
The module creates one table for the whois queries.
-
The module creates one table for the whois queries.
it didnt create the whois table for me, had to create it manually..
when i ran the whois-update.php it said it cant find redbotdb.whois table
when i ran the whoiscache.php it said it cant connect or something..
-
Strange.
The WhoisCache module contains a CREATE TABLE query, which should create the table.
And as the module is using the same functions all other bebot modules are using, it shouldn't create any errors.
-
0.3 svn has been updated to include Alreadythere's latest changes, and it has been moved into a core module.
The two versions are now almost fully interoperable, however some of the return values in the official version is different due to consistancy concerns.
-
I'm trying to add this to my bot, but Ive encountered a problem. I'm using Bebot_v0.2.6, and this is the first module I'm trying to add. I'm getting this error after running start.bat:
Fatal error: Call to undefined method Bot::define_tablename() in C:\BeBot2\modul
es\WhoisCache.php on line 12
Any ideas what I did wrong?
On a possible related note, I found no instances of get_site() in any of the php files. I'm using the "Find" command in notepad. Is this an example of my being a stupid noob at this? Or is it not unusual to not need to do any of those changes?
-
try reading http://bebot.link/index.php/topic,423.0.html (http://bebot.link/index.php/topic,423.0.html)
particularly you need to add the function define_tablename() to your Bot.php as Alreadythere mentioned there
Cheers
-
That's another of my functions for prefix and suffix for tablenames.
Add the following code behind the get_tablename function inside Bot.php, inside the Bot class:
function define_tablename($table, $pre, $suf)
{
return $table;
}
It will solve the error.
And be sure to add the get_settings() function mentioned in my first post too.
-
Thanks for the help guys, I appreciate your patience ;D
So I added the define_tablename function and put the module in the core folder. I also got a better handle on the correct syntax to search for, so was able to make the get_site changes as well. The bot starts fine now, but when I do a whois from a tell or in orgchat I get the error:
[2006-08-15 09:31:45] [TELL] [INC] Jaqueme: !whois cracknell
Fatal error: Cannot use string offset as an array in C:\BeBot2\core\WhoisCache.p
hp on line 68
C:\BeBot2>pause
Press any key to continue . . .
Did I miss a get_site entry?
Almost there...
-
I doubt that that error is in any relation to a get_site call.
I think it's an error or bug creaping up with arrays in some php version, because I have never seen that myself.
Sadly can't help you there myself, but I do remember it cropping up in some other modules too.
-
Might be this (http://bebot.link/index.php/topic,389.0.html)
-
Exactly that, yes.
Still using php4, so no experience there at all.
-
Ouch. Well, I tried re-building the bot with 4.1, but I get the php_sockets.dll not found error. The standard fix for this is to upgrade to php5.1.4, which is what is causing the problem in the first place. I tried editing the php.ini, but it returns the same error. I tried flying the .dll's over from php5.1.4, but it won't recognise them because they are from php5. And I am waaay too much of a noob to figure out how to code the corrections from the post Malosar referenced.
That sucks. Several of the modules I was interested in depend on the Whois Cache. The inability to communicate with IGN bots, however, is a severe limitation for my org. :-[
-
I've noticed the whois hasn't really been updating and the ingame command has been rather slow. So I checked the updatecache and found it was getting errors connecting to the AO site to pull down org member information. It was referencing community.anarchy.. etc however the main website references www.anarchy.... for the complete url to the xml of the org listing. So I changed it to that and it works fine.
btw if you plug the complete url with community.anarchy... etc then it times out with some python errors on the Ao site.
-
Thanks for that hint, modified the update script linked in the first post.
-
Seems either cname is having problems responding at times. I think FC might have put in temp blockers based on requests/sec possibly. I'm getting failures every night now. I'm increasing delaytime between requests from 100 to 1000msecs and see how it goes.
-
Ok, I've been using this for a while (love it by the way) and have done the adding stuff to the bot.php for the get_tablename stuff and everything, but not I'm wondering how you do it with multiple bots using one table (can either access from other bots database or setup a database for just the whois table and link them all, I don't care either way).
How exactly do I do that? Been looking over the module and everything and I am just lost here
-
I've got all my bots using one database, so sharing the whois table is easy for me. My get_tablename("whois") simply returns "whois" in all bots. Of course one database only works for me as I have modified every single database call in the modules I use to get the names with get_tablename().
If you already got several bots running in different databases, you can just return dbname.whois, with dbname the name of the database containing your whois-cache.
If you only got one bot running yet I'd advise you to switch to bebot 0.3 branch, which does support prefixes for tablenames in default. The scheme used there is just slightly different to mine, but it will be supported in the future.
-
It's really easy if you have bots with mutiple databases.
I created an aoglobal database for whereis and whois information.
All you need to do is grant your bot's MySQL username access to your aoglobal database, and edit Lines 60 and 104 in WhoisCache.php.
60: $lookup = $this -> bot -> db -> select("SELECT * FROM aoglobal.whoisRK1 WHERE nickname = '" . $name. "'");
104: $this -> bot -> db -> query("REPLACE INTO aoglobal.whoisRK1 (nickname, firstname, lastname, level, gender, breed, faction, profession,"
Some good ways to populate your whois cache quickly:
Run a large public raid bot. :p
Modify your tower attack module to use whois cache for whois data.
-
Yeah, my tower attack module is already setup to use it. I have been using it for a while (have just over 56,000 entries in mine) but not wanting to use it for mutliple bots to save space/update time ;)
This isn't working though. Now when something tries to access the database it tries bebot.aoglobal.whois. Its just appending the aoglobal.whois onto the bots current database so it isn't finding anything.
-
This isn't working though. Now when something tries to access the database it tries bebot.aoglobal.whois. Its just appending the aoglobal.whois onto the bots current database so it isn't finding anything.
Did you replace aoglobal with the database in which your whois cache is?
-
I made a database called aoglobal and put the whois table in there. When I made the change that was suggested it now tries BeBot.aoglobal.whois instead of just aoglobal.whois.
Could it be because I added the info into the bot.php that you listed on the first page for the get_tablename? I didn't think it would be linked to that since with the change that Glarawyn suggested it no longer references that function but I guess I could be wrong.
-
I'm trying to set this up with the .bat file but when i run the update.bat it display the text insted off execute it. even when i run the php w_update.php it only display the text insted off execute it.
I'm totally lost here :-[
-
Did you setup the correct info in the whois-update.conf? It dosen't use the main bot settings, it uses whatever you have in there for the update.
-
that was the first thing i did
-
oki i found a way around it now :)
just needed to do <?php insted of <?
and then add
extension=php_mysql.dll
extension_dir = "./"
as Alreadythere mentioned earlier
-
oki i found a way around it now :)
just needed to do <?php insted of <?
Guess you got strict mode (or whatever the name is) activated.
But going to change the tag in the next version I post, should be full tags anyways.
-
hmm now i get annoyed over this, when i try to run to update file i get this error
oki first i got this error:
PHP Fatal error: Maximum execution time of 60 seconds exceeded in C:\php\w_update.php on line 73
line 73:
mysql_query($query) or die("Query failed : " . mysql_error());
then i tryed again and got this error insted
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 1
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 1
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 1
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 1
Query failed : Out of range value adjusted for column 'org_rank_id' at row 1
line 19:
$tmp = explode("</" . $tag . ">", $tmp[1]);
???
-
This toon with ' in lastname caused the whois update to fail: http://auno.org/ao/char.php?dimension=1&name=Lavisha
Fixed by adding lines in bold to the manual whois-update.php code.
line 41~
$org = explode("<member>", $org);
$org = str_replace("'", " ", $org);
line 91~
$content = get_site("http://www.anarchy-online.com/character/bio/d/" . $dimension . "/name/" . strtolower($user['name']) . "/bio.xml");
$content = str_replace("'", " ", $content);
-
oki first i got this error:
PHP Fatal error: Maximum execution time of 60 seconds exceeded in C:\php\w_update.php on line 73
line 73:
mysql_query($query) or die("Query failed : " . mysql_error());
You have to set max execution time for your php parser I think (guess I should add it to the file). In many default settings php is only supposed to serve as webserver plugin. To stop bad code from running indefinitly php has a mechanism to abort after some pre-defined max execution time.[/]
I'll adapt the script as soon as I find time.
(http://de.php.net/manual/en/function.set-time-limit.php)then i tryed again and got this error insted
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 1
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 1
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 1
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 1
Query failed : Out of range value adjusted for column 'org_rank_id' at row 1
line 19:
$tmp = explode("</" . $tag . ">", $tmp[1]);
???
Sadly got to admit I don't have a clue here, never had that problem myself.
-
This toon with ' in lastname caused the whois update to fail: http://auno.org/ao/char.php?dimension=1&name=Lavisha
Fixed by adding lines in bold to the manual whois-update.php code.
line 41~
$org = explode("<member>", $org);
$org = str_replace("'", " ", $org);
line 91~
$content = get_site("http://www.anarchy-online.com/character/bio/d/" . $dimension . "/name/" . strtolower($user['name']) . "/bio.xml");
$content = str_replace("'", " ", $content);
Never knew that you could use ' in names, so never bothered to escape the strings.
Best would be to mysql_escape_string() all strings in the update script as well as when adding entries in the bot module. Then when returning character information use stripslashes().
-
* Best would be to mysql_escape_string() all strings in the update script as well as when adding entries in the bot module. Then when returning character information use stripslashes().
That's for your next version then Alreadythere :-)
-jj-
-
Updated all files, links in first post:
- set_time_limit(0); set, as long as save_mode isn't enabled the script won't be aborted due to too long execution time
- update-script supports used defined addons after all updates are done, use the file whois-update.addon. As it's included it will create a warning if no file of the name exists, but it won't abort. Creating an empty file of the name will remove the warning.
- mysql_escape_string() and stripslashes() added to remove errors with strange characters in names. First name, last name and org names are escaped.
- Added some more information to the output when the update script is done
- Modified the cache to use the same field names and order of fields in the table as the version shipped with the 0.3 branch. This is a purely database change, the names in the returned array haven't been changed. So no further modification to bebot files are needed besides replacing the WhoisCache.php. To avoid wrong behaviour of the cache you need to update the table! The SQL code below will do the actual updating, it's in a comment in the WhoisCache.phps file too. Replace whois with the name of your table. Depending on the size of the cache those update commands may take a few minutes to run.
ALTER IGNORE TABLE whois ADD id int(15) NOT NULL default '0' FIRST ;
ALTER IGNORE TABLE whois CHANGE name nickname VARCHAR(20) NOT NULL;
ALTER IGNORE TABLE whois ADD pictureurl VARCHAR(100) NOT NULL AFTER org_rank_id, ADD used BIGINT(25) NOT NULL AFTER pictureurl;
ALTER IGNORE TABLE whois ADD INDEX (used);
ALTER IGNORE TABLE whois ADD INDEX (updated);
-
* Best would be to mysql_escape_string() all strings in the update script as well as when adding entries in the bot module. Then when returning character information use stripslashes().
That's for your next version then Alreadythere :-)
Lol, was just posting the update ;)
-
Added line to add the ID field, missed that the old posted version didn't have that field yet.
ALTER IGNORE TABLE whois ADD id int(15) NOT NULL default '0' FIRST ;
-
oki it is coming when i have one in my db there are not in an org
made a temporally fix where i set the org_rank_id to 7 if they are not in an org but it still comes with the error 4 times in a row
-
Just stumbled over an old error report with a similar problem:
It seems some mysql setups are more strict concerning field types, so '' won't be accepted as a number.
Both .phps files updated with a simple check - if the id is empty it gets set to 0. This is done for org_id, rank_id and at number to be save.
Just redownload the .phps files, should solve your problem.
-
So anyone have a clue as to what I'm doing wrong with trying to get them all in one database? I still can't figure out why its just appending the database I put for the file onto my bots main database (instead of aoglobal.whois its trying to use bebot.aoglobal.whois).
-
conf file:
$tablename = "cache.whois"; // name of the whois table
php file:
$lookup = $this -> bot -> db -> select("SELECT * FROM cache.whois WHERE nickname = '" . $name . "'");
etc etc (just took out manually the table lookup stuff as I'm running slightly older bebot)
that's how mine is setup and works fine for 3 bots.
-jj-
-
i tryed to download the updated files but i get to many error and problems with it so for now i'll just stay with the old one, till i can find out of installing mysql better (or there come a idiot proof install guide) ;D
-
in my whois-update.conf file I have:
$dbname = "aoglobal";
$tablename = "whois";
In theWhoisCache.php I have this:
$lookup = $this -> bot -> db -> select("SELECT * FROM " . aoglobal.whois . " WHERE name = '" . $name . "'");
Slightly different than yours in the php file. I'll try making the small change to what you have and see if it makes a difference (looking at it it should be the exact same thing but I guess the bot might be interpreting it differently for some reason).
-
In theWhoisCache.php I have this:
$lookup = $this -> bot -> db -> select("SELECT * FROM " . aoglobal.whois . " WHERE name = '" . $name . "'");
I believe the aoglobal.whois has to be inside the "" too.
-
Dabaron you have to put it in this way
$lookup = $this -> bot -> db -> select("SELECT * FROM aoglobal.whois WHERE name = '" . $name . "'");
for it to work
-
New version seems to be crashing if it doesn't find someone in database. Cannot use string as an offset, bot dies...
got round it by only doing the $whoold if something is found in lookup but this kinda defeats the new code...
if (!empty($lookup))
{
$this -> bot -> log("WHOIS", "STATUS", "entry found in lookup database");
$whoold["nick"] = $lookup[0][1];
-
Updated my version, same link in first post.
Basically I'm doing the same as you, I've pulled the (!empty($lookup)) out of the stale test to the front. If it's not in the table yet it doesn't need to be checked for outdated info anyways.
-
Thanks for quick help :-)
-jj-
-
oki gone over to the updated version now but when i do a update i get
PHP Warning: include<whois-update.addons>: failed to open stream: No such file or directory in c:\php\w_update.php on line 197
PHP Warning: include<>: Failed opening 'whois-update.addons' for inclusion <include_path='.;c:\php5\pear'> in c:\php\w_update.php on line 197
did i do something rong with the musql and|or php again ???
-
That worked. Thanks guys, you ROCK
-
oki gone over to the updated version now but when i do a update i get
PHP Warning: include<whois-update.addons>: failed to open stream: No such file or directory in c:\php\w_update.php on line 197
PHP Warning: include<>: Failed opening 'whois-update.addons' for inclusion <include_path='.;c:\php5\pear'> in c:\php\w_update.php on line 197
did i do something rong with the musql and|or php again ???
No, not at all.
I just added an include "whois-update.addons" at the end of file for possible further processing (which I'm doing).
The warning just tells you that the file doesn't exist (which is no problem at all). If you want to get rid of it either remove the include line or create an empty whois-update.addons file. I'll try to get around and add an exist-check for the file before the include in the next version.
-
whois-update.php updated with an exist/readable check on the addons file.
Should remove the warnings even without the file.
-
oki thx
-
Updated lookup function with optional second parameter $nolookup, as per Wolfbiter's idea.
If you got some lookups that are don't need to check for updated information because they aren't that important do $lookup($name, true), and the cache won't try to get new information of the FC site even if the cached one is out-of-date.
Removed all references to get_tablename, get_setting and define_tablename functions, there is now one global parameter for the name of the whois table. Adapt that to your needs, it's supposed to be global over all bots anyways.
This should make it easier for new user to install the whois-cache, as no modifications to the Bot.php are needed anymore. Only the modules that use lookups have to be modified still.
Links in first post point to current updated files.
-
after applied the new whoiscache, i get this msg everytime a ppl log on or any tower attacks ;X
[2006-11-08 08:21:31] [GROUP] [MSG] [All Towers] 0: Wocci (CLAN, Komodo) attacked Outer Circle (OMNI) at Belial Forest, 521x350.
MySQL error (# 0) on query: UPDATED whois SET firstname = 'Me', lastname = 'Crush', level = '140', gender = 'Neuter', breed = 'Atrox', faction = 'Clan', profess
ion = 'Shade', defender_rank = 'Student', defender_rank_id = '6', org_id = '569345', org_name = 'Komodo', org_rank = 'Applicant', org_rank_id = '4', updated = '1162974092' WHERE nickname = 'Wocci'
-
If you had an older version of the cache running, did you modifiy the table scheme as mentioned in this post (http://bebot.link/index.php/topic,223.msg3405.html#msg3405)?
-
Error Code : 1060
Duplicate column name 'ID'
(0 ms taken)
Error Code : 1054
Unknown column 'name' in 'whois'
(0 ms taken)
Error Code : 1060
Duplicate column name 'pictureurl'
(0 ms taken)
(1 row(s) affected)
(0 ms taken)
(1 row(s) affected)
(0 ms taken)
Get this error msgs when I applied the SQL codes :/
If i remove the table and reboot bot to recreate the table, it will work till next time i reboot bot then it will give me the same errors again :(
-
Sorry, was an error in the code (UPDATED instead of the correct UPDATE).
Just redownload the file in first post, or edit the line manually. All you have to do then is remove the D at the end.
-
Added updated version (http://www.m8y.de/ao/bebot/whois/WhoisCache.phps.0.3) of the WhoisCache file in the 0.3 branche. The following updates or fixes were applied:
- Added optional $nolookup parameter to the function call. If $nolookup == true then no xml lookup will be done, even if the data is outdated or not existing.
- mysql_escape_string() and stripslashes() added to prevent errors with strange signs in names.
- Check for empty ids added, those will be set to '0' to prevent errors in some mysql setups.
File is also linked to in first post.
-
Updated the table structure. Main change is extensive use of enums instead of varchars where only a limited number of choices exist. Main effect is a reduction in the size of the database.
Updated files in first post, the working of the cache hasn't changed in any way.
To convert old tables to the updated structure use the following SQL code:
ALTER IGNORE TABLE whois
CHANGE ID ID int(15) NOT NULL default '0',
CHANGE nickname nickname varchar(15) NOT NULL default '',
CHANGE firstname firstname varchar(20) NOT NULL default '',
CHANGE lastname lastname varchar(20) NOT NULL default '',
CHANGE level level tinyint(3) unsigned NOT NULL default '1',
CHANGE gender gender enum('Female','Male','Neuter') NOT NULL default 'Female',
CHANGE breed breed enum('Atrox','Nano','Opifex','Solitus') NOT NULL default 'Atrox',
CHANGE faction faction enum('Clan','N/A','Neutral','Omni') NOT NULL default 'Clan',
CHANGE profession profession enum('Adventurer','Agent','Bureaucrat','Doctor','Enforcer','Engineer','Fixer','Keeper','Martial Artist','Meta-Physicist','Nano-Technician','Shade','Soldier','Trader') NOT NULL default 'Adventurer',
CHANGE defender_rank defender_rank enum('Able','Accomplished','Adept','Amateur','Backer','Beginner','Challenger','Champ','Common','Competent','Defender','Fair','Fledgling','Guardian','Hero','Intermediate','Medalist','Mediocre','Newcomer','None','Patron','Protector','Qualified','Starter','Student','Suited','Supporter','Talented','Trustworthy','Vanquisher','Vindicator') NOT NULL default 'Able',
CHANGE defender_rank_id defender_rank_id enum('0','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25','26','27','28','29','30') NOT NULL default '0',
CHANGE org_id org_id bigint(10) NOT NULL default '0',
CHANGE org_name org_name varchar(50) NOT NULL default '',
CHANGE org_rank org_rank varchar(20) NOT NULL default '',
CHANGE org_rank_id org_rank_id enum('0','1','2','3','4','5','6','7','8','9','10') NOT NULL default '0',
CHANGE pictureurl pictureurl varchar(100) NOT NULL default '',
CHANGE used used bigint(25) NOT NULL default '0',
CHANGE updated updated int(10) NOT NULL default '0'
Change whois to the name of your cache table.
-
forgive me......but I have been looking at this thread for the past couple of days.......and have not been able to get this to work.....is there a step by step install guide??? or can I see/get a list of all the files that need changing to get this to work??? I am currently using 0.2.9 with PHP 5.2.0 install.
thanks
bOogY
-
i'm starting to get an error again.
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 22
that line
$tmp = explode("</" . $tag . ">", $tmp[1]);
-
i'm starting to get an error again.
PHP Notice: Undefined offset: 1 in C:\php\w_update.php on line 22
that line
$tmp = explode("</" . $tag . ">", $tmp[1]);
It's prolly producing this when trying to parse a missing org or player information.
It whouldn't produce any errors though, just empty return strings as the array used doesn't contain a second element.
The notice is nice if you have to find some out-of-bound array index, but in this case it won't produce any wrong behaviour, empty information is checked after the parsing.
-
forgive me......but I have been looking at this thread for the past couple of days.......and have not been able to get this to work.....is there a step by step install guide??? or can I see/get a list of all the files that need changing to get this to work??? I am currently using 0.2.9 with PHP 5.2.0 install.
Basically you have to edit every module that uses get_site to read character information of FC.
$this -> bot -> get_site("http://www.anarchy-online.com/character/bio/d/" . $dimension . "/name/" . $user . "/bio.xml");That's every line similar to the one above.
Out of my head modules that need to be edited are:- whois
- tower-alert
- teams
- online
- logon
I could have missed a couple still.
-
SVN contains a newer version of this module for the 0.3 branch, which Glarawyn has extended to offer an additional fallback to AUNO if FC doesn't offer valid information. Everyone testing the 0.3 branch should grab that module!
-
As posted here (http://bebot.fieses.net/viewtopic.php?t=220) I am using a cache for all whois entries.
Link not working. ^^
Edit: no, wait, I think I figured it out. Nvm.
-
Link corrected anyways :)
-
Ok, trying to get this to work on my bot, but when i try to fetch the db (when running php4 whois-update.php) i get the error:
"Query failed : Table 'aodb.whois' doesn't exist"
my database is called aodb, but it dosnt seems like the script adds any tables to it when i run it... anyone got any idea of what might be wrong?
-
First check whether you do actually have a whois table in your database. Second, check the console for errors on startup when it is trying to create the table.
-
ok, i assumed that the script would create the tables for me, since most ppl know how to handle mysql (including me), and everything else was done by the script.. :/
so the script dosnt create the tables in the database, you have to do it manualy?
-
The whois-update script doesn't create the table. The WhoisCache.php module creates the table.
-
AHHH, tyvm then, i'll try to get it running then :]
-
stupid question I know, but how do I run a php script outside the bot?
What program? Or as I assume I use php.exe via cmd prompt but how?
Thanks in advance
-
php blahblah.php
It's as simple as that. What you would want to do however is make a batch file to run the updater and add that updater to a scheduler to run at a specific time every day.
-
You need to call php with the script as parameter.
Should be similar to the following (replace PathToPhp with the correct path):
PathToPhp\php whois-update.php
It needs some entries in the database to work with though :)
-
ok I did what you said.
A dos prompt window appears and goes away in about 1 second. Is this correct?
Edit: Whois table is already in the db as I have been using the modififed whois and cache since I first switched to BeBot 3 weeks ago.
-
Go to a command prompt first, don't run it via start -> run. That way if you get errors you'll be able to see them without the window just closing immediately. And make sure your running the update script in the command console, not the module itself :)
Also the update lakes a long time. I have my delay set at 1000ms and it takes about 3 hours to complete (but I also have 85k lines in my whois db).
-
Yes it appears that once I set it up in a batch fle with PAUSE I caught my errors.
I had to update my php.ini file and then it worked.
However, FC is having issues as a response from them timed out!! I went to the forums and it took forever to load and sometimes timed out.
It is running right now, so I guess it will just take the 3hours as you said and who knows since it seems FC is crashing.
Thanks!!!!
-
i get a somewhat strange error:
"Warning: fopen(http://www.anarchy-online.com/org/stats/d/rk1/name/2232322/basicstats.xml): failed to open stream: HTTP request failed! HTTP/1.0 500 Internal Server Error
in /home/bebot/whois-update.php on line 10
Warning: fgets(): supplied argument is not a valid stream resource in /home/bebot/whois-update.php on line 13"
when i try to run the script...
/* Line */
/* 08 */ unction get_site($url)
/* 09 */ {
/* 10 */ $fp = fopen($url, "r");
/* 11 */ $content = ""; //11
/* 12 */
/* 13 */ while ($buffer = fgets($fp, 1024))
/* 14 */ $content .= $buffer;
/* 15 */
/* 16 */ return $content;
/* 17 */ }
the line notice isnt actually in the script, just for this post :)
-
I get a few of those as well. Like 10-50 when updating over 50K entries. Was guessing that FC site was slow to respond or something like that.
-
Those are just warnings, no errors.
There are two possible reasons for this behavior:
1) the character or org doesn't exist.
2) FCs server is slow.
I'm just ignoring those warnings as the cache and updating still works.
-
================================================
739 http queries for org information done!
47138 total http queries done!
42216 entries in database associated with an org modified!
87207 entries in database modified!
88570 entries in the whois cache!
232min 51sec runtime for org updates!
638min 24sec total runtime!
================================================
lol, I think "the internet" was having issues :D
-
I run mine before work, done by the time I get home :D
-jj-
-
wow, for 40k entries in my DB takes about 27mins to run.
638min runtime... just wow! I think max I have had is 35 mins for 40k entries!
-
Well I have my delay set at 1000ms instead of 100ms, but thats a lot longer than usual.
Normal day:
26min 31sec runtime for org updates!
75min 24sec total runtime!
-
Hm, I got like 80mins for org and 440mins total update time usually, with 110k total entries, 80k of which are orged.
-
Ported the 0.2.10 get_site function to whois-update.php.
Now the script doesn't produce endless warnings due to timeouts on the http queries anymore.
Just replace the old script with the new version, linked in the first post.
-
I'm still having some issues running the w_update.php.
I configured the whois-update.conf and have it in the same directory as my w_update.php file. When I run the php file I get:
Fatal error: Call to undefined function mysql_connect() in C:\BeBot\w_update.php
on line 117
----------
Line 117: $link = mysql_connect("localhost", $username, $password)
----------
select COUNT(*) from AOBot' at line 1
mysql> select COUNT(*) from whois;
+----------+
| COUNT(*) |
+----------+
| 2 |
+----------+
1 row in set (0.00 sec)
Any thoughts?
Bryan
-
I made a change to my php.ini file that included:
extension = php_mysql.dll
extension_dir = ./php_extension/
When the script is run the following is returned:
'extension' is not recognized as an internal or external command,
operable program or batch file.
================================================
2 http queries for org information done!
2 total http queries done!
295 entries in database associated with an org modified!
295 entries in database modified!
295 entries in the whois cache!
0min 7sec runtime for org updates!
0min 7sec total runtime!
================================================
Am I doing something wrong? After reading through this post it appears that most people have over 40k records returned and I'm only returning 295.
Thanks in advance.
Bryan
-
It depends on what modules you are running. If you just run the updated relay_guild, this will check only the cached data when someone joins the privategroup.
The way the whois cache works is that whenever it is queried, it will add an entry if it isn't found (downloaded the old way from FC site). When you manually update the cache via that script instead of updating each single person, it will download the entire org data for each org it sees and add each person from that org.
So to fully populate it, you should update a few other plugins to take advantage of it. You can look here (http://bebot.link/index.php/topic,627.0.html) for a mostly complete list of available plugins. Ones that (imo) you should update would be the whois module and the towerattack module as these take great advantage of the caching features. Once you have installed the whois update, then !whois one person from each org you can think of and run your update script again, it will soon fill up to a large level :-) You should also notice a nice speed boost in the bot whenever the whois cache is utilised (especially with !whois requests as previously this could take a long time if the FC site is slow).
Let me know if you need any further information.
Cheers,
-jj-
-
Not sure what's up with the extension for you. It could be though that you don't need it - that's the case if mysql is compiled into your php.exe.
The low number of entries results most likely of just a few entries in your cache. For the exact description read jjones666's post, he was a few mins faster then me :)
-
Btw, on the php.ini issue, here is mine:
; Error handling and logging ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
display_errors = On
display_startup_errors = On
log_errors = On
log_errors_max_len = 0
error_log = ./log/php_errors.txt
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; BeBot will normally use about 10-13megs with default
; modules. Default php memory limit is 8megs.
; Raise default on hosts that permitt it.
memory_limit = 20M
; Mainly for windows where additional modules are always required
extension_dir = ./php_extension/
I'm not sure of any need to put that extension line in if your whois_update.php and whois_update.conf files are in the root directory of your bot. The error handling on the first few lines isn't essential, just useful.
Edit:
I'm assuming your directory structure is bebot standard and you do have a directory called php_extension in the root (which contains the needed php_mysql.dll file along with some others).
-jj-
-
Hey Guys,
Thanks for the responses. The extension error was because I had a brain fart last night and added the @extension = php_mysql.dll to my batch file.
But now I have another question regarding caching TowerAttack and Whois. Ive placed the updated php files in my bots modules directory but Im not sure what to do to force the TowerAttack module to update.
Also, after restarting the bot the log returns these errors:
MySQL error (# 0) on query: SELECT nickname, level, rank_name, profession, id, a
ilevel FROM members WHERE id = 590021014
Unknown column 'ailevel' in 'field list'
MySQL error (# 1) on query: UPDATE members SET lastseen = 1169584767 WHERE nickn
ame = 'Cero'
Unknown column 'lastseen' in 'field list'
MySQL error (# 2) on query: SELECT nickname, level, rank_name, profession, id, a
ilevel FROM members WHERE id = 925463630
Unknown column 'ailevel' in 'field list'
MySQL error (# 3) on query: UPDATE members SET lastseen = 1169584767 WHERE nickn
ame = 'Goldfist'
Unknown column 'lastseen' in 'field list'
MySQL error (# 4) on query: SELECT nickname, level, rank_name, profession, id, a
ilevel FROM members WHERE id = 1575319688
Unknown column 'ailevel' in 'field list'
MySQL error (# 5) on query: UPDATE members SET lastseen = 1169584769 WHERE nickn
ame = 'Alphaheals'
Unknown column 'lastseen' in 'field list'
MySQL error (# 0) on query: SELECT id, rank, level, ailevel, lastseen, updated F
ROM members WHERE id = 814199987
Unknown column 'ailevel' in 'field list'
MySQL error (# 1) on query: REPLACE INTO members (id, nickname, firstname, lastn
ame, rank, rank_name, level, profession, gender, breed, ailevel, aititle, lastse
en, updated)
VALUES ('814199987',
'Bloodstain', '',
'', '0',
'President', '216',
'Engineer', 'Male',
'Nano', '15',
'Competent', '0', '1169584814')
Does anyone know what would cause this?
Regards,
Bryan
-
I'd guess you are using some updated Members module, but forgot to add the new fields to your members table.
-
Or your using new towerattack and whois components but haven't updated your roster and thus don't have the required fields.
-
Hi,
I am using the new using the new towerattack and whois module and I did update the roster so Im not sure whats wrong now. I dont get these errors with the default php files. If I delete the members table will it be automatically recreated?
Bryan
-
Roster_GUILD.php will rebuild and populate the members table if members table is deleted.
-
I configured the whois-update.conf and have it in the same directory as my w_update.php file. When I run the php file I get:
Fatal error: Call to undefined function mysql_connect() in C:\BeBot\w_update.php
on line 117
----------
Line 117: $link = mysql_connect("localhost", $username, $password)
[/quote]
umm having the same problem here.. What did you find out?
Edit:
Looks like it dont like php5...used an older version.. one that came with .2 bot.. and it kind of works..
Warning: mysql_escape_string(): This function is deprecated; use mysql_real_esca
pe_string() instead. in C:\aobots\rsbot\aofbot\whoisupdate\whois-update.php on l
ine 224
Warning: mysql_escape_string(): This function is deprecated; use mysql_real_esca
pe_string() instead. in C:\aobots\rsbot\aofbot\whoisupdate\whois-update.php on l
ine 234
================================================
3 http queries for org information done!
4 total http queries done!
1265 entries in database associated with an org modified!
1266 entries in database modified!
1266 entries in the whois cache!
0min 37sec runtime for org updates!
0min 38sec total runtime!
================================================
-
Yep, I have same issue, it's no concern really tho as I prefer having the whois updating script in a different directory running with PHP4, while the bot is in main directory running PHP5.
-jj-
-
What's the problem exactly? I don't have php5 anywhere yet, so I don't know what I have to change to get it running with it.
I'll replace mysql_escape_string with mysql_real_escape_string in the next version to remove the deprecated warning.
-
works great to me :D even moved from windows to linux and no problems :D
-
Processing Update ...
Update began at Tue Feb 27, 2007 12:47 am
================================================
260 http queries for org information done!
1995 total http queries done!
54333 entries in database associated with an org modified!
55969 entries in database modified!
55969 entries in the whois cache!
31min 15sec runtime for org updates!
40min 17sec total runtime!
================================================
Update concluded at Tue Feb 27, 2007 1:27 am
I edited the update script to output to a directory so I can see the changes. But it runs in PHP 5.2.1 and I don't get any errors ???
-
I configured the whois-update.conf and have it in the same directory as my w_update.php file. When I run the php file I get:
Fatal error: Call to undefined function mysql_connect() in C:\BeBot\w_update.php
on line 117
----------
Line 117: $link = mysql_connect("localhost", $username, $password)
You seem to be missing the mysql library for your php5 version. Looks like it isn't statically linked, so you'll have to add a php.ini file for it.
-
Ive just recently started using BeBot as main guild chat bot, and so far after added a lot of modules Im starting to like it, however Ive got this little problem with the whois-cache.
C:\Program Files (x86)\BeBot_v0.2.11\whoiscache>php.exe w_update.php
PHP Notice: Use of undefined constant host - assumed 'host' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 18
PHP Notice: Use of undefined constant path - assumed 'path' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 37
PHP Notice: Use of undefined constant query - assumed 'query' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 38
PHP Notice: Use of undefined constant timed_out - assumed 'timed_out' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 61
PHP Notice: Use of undefined constant timed_out - assumed 'timed_out' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 83
PHP Notice: Use of undefined constant host - assumed 'host' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 18
PHP Notice: Use of undefined constant path - assumed 'path' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 37
PHP Notice: Use of undefined constant query - assumed 'query' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 38
PHP Notice: Use of undefined constant timed_out - assumed 'timed_out' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 61
PHP Notice: Use of undefined constant timed_out - assumed 'timed_out' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 83
>snip<
PHP Notice: Use of undefined constant timed_out - assumed 'timed_out' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 83
PHP Notice: Undefined offset: 1 in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 112
PHP Notice: Undefined offset: 1 in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 112
PHP Notice: Undefined offset: 1 in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 112
PHP Notice: Undefined offset: 1 in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 112
================================================
5 http queries for org information done!
8 total http queries done!
1090 entries in database associated with an org modified!
1092 entries in database modified!
1093 entries in the whois cache!
1min 9sec runtime for org updates!
1min 26sec total runtime!
================================================
C:\Program Files (x86)\BeBot_v0.2.11\whoiscache>
while it does look like it completes the script the PHP Notice: spam is huge and it grows in size everytime the cache grows so Im starting to wonder if its healthy ;)
$username = "*******"; // the user name for accessing the database server
$password = "*******"; // the password to access the database server
$dbname = "********"; // the database to use
$dimension = "1";
$dbserver = "localhost"; // the server the DB runs on, localhost in most cases
$tablename = "whois"; // name of the whois table
$thistime = time();
$hours = 36; // after how many hours without an update delete an existing entry?
// used to get rid of stale entries of non-existing/no longer existing characters
$delaytime = 1000; // microseconds to wait after each parse step to reduce load on the http server of FC
[code]
here's my whois-update.conf file and Im using the files linked to in the first post with php version 5.2.1
would this 'problem' go away if I used for example a php4 exe on the update script?[/code]
-
here's my whois-update.conf file and Im using the files linked to in the first post with php version 5.2.1
would this 'problem' go away if I used for example a php4 exe on the update script?
Tbh I don't have any clue what's happening.
If you can, try it with some other php version, I'm still using 4.3 here.
It could be some oversensitive settings in your php ini too though which result in all notices spammed.
I'll try to get around to check those lines in the notices tomorrow, perhaps it helps.
-
PHP Notice: Use of undefined constant host - assumed 'host' in C:\Program Files (x86)\BeBot_v0.2.11\whoiscache\w_update.php on line 18
If you have a look at line 18 of that script you will probably see an array like $myArra[host] which should have been $myArray['host']
These are notices and not errors nor even warnings and the update should function if these are the only thing "wrong".
To not have to see these notices adjust your php.ini to supress notices (ie. just show errors and warnings).
But "the right way" to fix it is to use strings in array indexes instead of undefined constants. (ie. $myArray['host'] in stead of $myArray[host])
Hope that helps.
-
If you have a look at line 18 of that script you will probably see an array like $myArra[host] which should have been $myArray['host']
[...]
But "the right way" to fix it is to use strings in array indexes instead of undefined constants. (ie. $myArray['host'] in stead of $myArray[host])
Hope that helps.
Nice analysis there, was exactly that.
I've updated the script on the first page, it should fix those notices (except the one about no offset of 1 in line 112, not sure yet if I'll try to catch that).
-
Ive downloaded the updated update script from the first page and yes, it works better now.
I'm still getting that "PHP Notice: Undefined offset:" 1 in line 112 though BUT!
================================================
10 http queries for org information done!
15 total http queries done!
3115 entries in database associated with an org modified!
3119 entries in database modified!
3120 entries in the whois cache!
2min 9sec runtime for org updates!
2min 13sec total runtime!
================================================
================================================
10 http queries for org information done!
15 total http queries done!
3115 entries in database associated with an org modified!
3119 entries in database modified!
3120 entries in the whois cache!
1min 16sec runtime for org updates!
1min 19sec total runtime!
================================================
First one is from the old script that spammed me all those messages, last one is the new script that only gives me that offset of 1 in 112 message.
notice any diffrence? ;D
-
It runs faster :)
Which could just be due to less delay for the output though.
-
Keep in mind that I don't know what the code is. If someone can post the relevant lines (ie 105-115 I might be able to debug it. ;)
Edit: Tyop
-
function xmlparse($xml, $tag)
{
$tmp = explode("<" . $tag . ">", $xml);
$tmp = explode("</" . $tag . ">", $tmp[1]);
return $tmp[0];
}It's the $tmp[1] here, which prolly creates an notice if $tag doesn't exist in the $xml var. Adding a simple check like the following may perhaps solve the notice without breaking anything:
function xmlparse($xml, $tag)
{
$tmp = explode("<" . $tag . ">", $xml);
if (!(isset($tmp[1])))
return "";
$tmp = explode("</" . $tag . ">", $tmp[1]);
return $tmp[0];
}
EDIT: Those two functions are direct copies from the Bot class btw, that should create the same notices then too.
-
Updated the whois-update.php file in first post.
- Added workaround for the notice in the xmlparse() function
- updating pictureurl now too, no idea why I forgot that
- optimized handling of org members a bit
-
Alreadythere,
Line 6 of the individual character DB update should be:
. "', org_rank_id='" .$who["rank_id"]. "', updated='" .$thistime. "', pictureurl='" .$who["pictureurl"]. "' WHERE nickname='" .$who["nick"]. "';";
(Delete line 7).
Explode for org only should be:
$who["pictureurl"] = xmlparse($content, "photo_url");
I think anyway :-)
-jj-
-
I believe you are right, thanks.
Updated :)
-
I'm pretty new to Be-Bot have been fine till this module upgrade :/ I get:
PHP Fatal error: Cannot redeclare class WhoisCache in C:\Documents and Settings\Jeremy\My Documents\BeBot_v0.2.11\modules\WhoisCache.php on line 71
when i run bot now:( Any suggestions or help?
-
The "Cannot redeclare class" error message occures when you attempt to declare a class twice. This is ofcourse not allowed.
The cause of this is in this case that you've got two modules that attempt to declare the class "WhoisCache".
The solution to this is to rename the old module that handled it to have an underscore (_) in front of it so that bebot doesn't attempt to load it.
If you put the new class in the custom directory you need to rename the one in the modules (or core if that's where it resides) directory.
Hope that helps.
-
I'm pretty new to Be-Bot have been fine till this module upgrade :/ I get:
PHP Fatal error: Cannot redeclare class WhoisCache in C:\Documents and Settings\Jeremy\My Documents\BeBot_v0.2.11\modules\WhoisCache.php on line 71
when i run bot now:( Any suggestions or help?
Variation of the above suggestion: If you edited the code in the modules directory (i.e. not before moving it there) your text editor may have saved a backup with some appendix to the name and therefore the bot tries to load it twice.
-
Variation of the above suggestion: If you edited the code in the modules directory (i.e. not before moving it there) your text editor may have saved a backup with some appendix to the name and therefore the bot tries to load it twice.
If the module doesn't end with exactly .php it should not be attempted loaded. Most *nix editors saves files as .php~ and they are not loaded, however if you manually saved it as .bak.php (ie. instead of .php.bak which would be "the right way(tm)) it will still attempt to load it and re-declear the class.
-
If the module doesn't end with exactly .php it should not be attempted loaded. Most *nix editors saves files as .php~ and they are not loaded
Oh good, I thought it was only looking at whether the first character in the filename was an underscore or not. Should have checked it first but I'd just discovered a backup file of the ~ kind there the other day and thought removing that file was what fixed my bot but in retrospect I am sure I did something else at the same time.
-
The last time i put this in my linux cronjob it crashed the server every time it executed (from the 3rd time onwards)
Any idea how to prevent this? i'd rather it didnt crash ::)
-
Uhm it shouldn't crash any system. I'm running the script daily since more then one year, and it never crashed my server.
You could try to increase the delay between http queries from 100 to 1000 milliseconds, maybe your server can't handle any heavy mysql load.
-
I'll try again then on the more powerful server
-
The only error i get (some time, not all the time) is fsocketopen or something like that, usually twice per run. Still can't reproduce it on the spot so can't really tell you much info about it. And that doesn't crash anything.. just doesn't update the database :P
Tony
-
Warning: fsockopen(): php_network_getaddresses: getaddrinfo failed: Temporary failure in name resolution in /home/aobots/scripts/whois-update.php on line 18
Warning: fsockopen(): unable to connect to www.anarchy-online.com:80 in /home/aobots/scripts/whois-update.php on line 18
I guess that's the errors you talk about tonyuh. It's some problem to get the name resolved and connect afterwards. Can happen I guess, and unless you get many of them quite harmless. There is a safety before any entries are deleted as state.
-
Modified the update script and it's config file, per default it now only pulls the org rosters of the FC server to avoid too high load. And it doesn't delete outdated entries at all anymore.
If you want the old behaviour back you'll have to change the new config options.
EDIT: This version works with the 0.4 cache too if you don't want to grab the SVN version.