I am just wondering if it's necessary to use utf8_encode/utf8_decode function in the Bebot code? Most MySql servers work fine with utf-8, all last versions of PhPMyAdmin use utf-8 now. AoC uses utf-8 as default. As utf8_encode/utf8_decode have limitation (as well as iconv or mbstring) and AoC uses uft-8 why dont you wanna just use utf-8 as default for bebot? uft-8 is a key here. utf8_encode/utf8_decode encodes or decodes an ISO-8859-1 string to UTF-8 (and vice versa) only. You are not able to convert ISO-8859-5 string to UTF-8 with utf8_encode. So BeBot has limitation for Russian, Bulgarian or Arabic users for example.

If you are afraid of non-utf8 configuration of MySQL why don't just add mysql_query("SET NAMES utf8"); to MySQL.php and delete all utf8_encode/utf8_decode functions. 

$conn = mysql_connect($this -> SERVER, $this -> USER, $this -> PASS);
mysql_query("SET NAMES utf8");  //using UTF-8 charset
if (!$conn)
I am reading a lot of russians forums with bebot messages and the most common question is how to make bebot works with russian characters (means utf-8).

Is you are not willing to change bebot to utf-8 let me start a new google code page with bebot utf-8 support course I am kind tired of explaining to every russian guild in AoC how to set Bebot with utf-8 support . But I dont wanna start it unless I have a permition from you guys.

Bebot is a great product and a big help to all AO and AoC games so let's make this product available to all. Thanks a lot.

Starting a 2nd project off a project that is based on another project is a bit silly.

We are having some discussions on how we can merge the two current projects into one. I would wait a bit with the utf8-stuff.

That's right, it's silly to start another project (as it's need just a few things to change) that's why I start this topic.

So I'll wait a bit with utf-8 as I know you have a lot of other things to do. Thanks for the replay. If you need any test with utf-8 (non-latin) let me know I am willing to help as I have SentOS Linux server with MySQL (4.1 could switch to 5) and PHPMyAdmin (4.1.22-standard), PHP (5.2.6).

Ok, first of all, let me explain a few things.

Mostly, by default MySQL servers are set to use latin1 characters (if you use PhpMyAdmin you can easily check it by pressing Show MySQL system variables link).

If you run your own MySQL server you may easily configure your server to use pure utf8 for server's character set and collation instead of default latin1. This will help to correctly store non-latin character data in db (cyrillic ?haracters for example and russian language in particularly). Go to the [mysqld] section in my.cnf and add two strings:

collation_server=utf8_general_ci
character_set_server=utf8
You can also add

skip-character-set-client-handshake
to enforce using of utf8 encoding in db.

I dont know anything about MySQL on Windows machine but I am sure it works the same way as on Linux one.

But most of us just use some kind server providing hosting service to run BeBot and we are not able to make any setting changes in MySQL. So the way above dont work . If you have access to phpMyAdmin check if your MySQL charset set as: UTF-8 Unicode (utf8). If it sets as UTF-8 (and MySQL connection collation: utf8_general_ci) you are lucky and your provider cares about all non-ASCII characters users. If you remove all utf8_encode/utf8_decode function in BeBot code your non-ASCII characters will work fine with MySQL (tested by my AoC guild for more that two weeks). But even this way your stored non-ASCII characters show incorrectly in phpMyAdmin and you are not able to make any changes in MySQL manually if you need.

So, we came to conclusion, two ways above are not ok for us as we need to find something different, very simple to install.

As we want to make it easy to install for user we need to make some changes in MySQL query in BeBot php code.

1. Source\MySQL.php (bebot-hyborian.0.5.2.r4)

creat tables with default charsets utf-8:

---   91 $this -> query("CREATE TABLE IF NOT EXISTS " . $this -> master_tablename . "(internal_name VARCHAR(255) NOT NULL PRIMARY KEY, prefix VARCHAR(100), use_prefix VARCHAR(10) NOT NULL DEFAULT 'false', schemaversion INT(3) NOT NULL DEFAULT 1)");
---  92 $this -> query("CREATE TABLE IF NOT EXISTS table_versions (internal_name VARCHAR(255) NOT NULL PRIMARY KEY, schemaversion INT(3) NOT NULL DEFAULT 1)");

 
+++  91 $this -> query("CREATE TABLE IF NOT EXISTS " . $this -> master_tablename . "(internal_name VARCHAR(255) NOT NULL PRIMARY KEY, prefix VARCHAR(100), use_prefix VARCHAR(10) NOT NULL DEFAULT 'false', schemaversion INT(3) NOT NULL DEFAULT 1) DEFAULT CHARSET=utf8");
+++ 92 $this -> query("CREATE TABLE IF NOT EXISTS table_versions (internal_name VARCHAR(255) NOT NULL PRIMARY KEY, schemaversion INT(3) NOT NULL DEFAULT 1) DEFAULT CHARSET=utf8");
and between:

       124 $conn = mysql_connect($this -> SERVER, $this -> USER, $this -> PASS);
       125
       126 if (!$conn)
insert:

+++ 125 mysql_query("SET NAMES `utf8` COLLATE `utf8_general_ci`");

2. As we are gonna use UTF-8 we are no longer need utf8_encode/utf8_decode function as it incorrectly convert non-ASCII characters, so we need just comment all these strings:

15_ChatQueue.php

Line 89 : //$msg = utf8_encode($msg);
Line 94 : //$msg = utf8_encode($msg);
Line 115 : //$msg = utf8_encode($msg);
Line 120 : //$msg = utf8_encode($msg);
Sources\Bot.php

Line 589 : //$msg = utf8_encode($msg);
Line 632 : //$msg = utf8_encode($msg);
Line 684 : //$msg = utf8_encode($msg);
Line 958 : //$args[1] = utf8_decode($args[1]);
Line 1183 : //$args[2] = utf8_decode($args[2]);
Line 1263 : //$args[2] = utf8_decode($args[2]);
That's all folks! BeBot with non-ASCII characters support works great for my guild.

P.S. UTF-8 is a future as it can hold more than 100 000 characters. Compare it with ISO-8859-1 (it holds only 256 characters).