collapse collapse
* User Info
 
 
Welcome, Guest. Please login or register.
* Search

* Board Stats
  • stats Total Members: 989
  • stats Total Posts: 18363
  • stats Total Topics: 2500
  • stats Total Categories: 7
  • stats Total Boards: 35
  • stats Most Online: 1144

Author Topic: SQLite  (Read 3495 times)

0 Members and 1 Guest are viewing this topic.

Offline Tutomech

  • BeBot Rookie
  • *
  • Posts: 12
  • Karma: +0/-0
SQLite
« on: July 07, 2010, 02:32:19 pm »
I could not find any related posts.

But was there ever SQLlite or any other in process DB considerations.
I would love not needing a DB service to run on my host.

Offline Khalem

  • BeBot Founder
  • Administrator
  • ********
  • Posts: 1169
  • Karma: +0/-0
    • http://www.ancarim.com
Re: SQLite
« Reply #1 on: July 08, 2010, 02:17:43 pm »
sqlite support is something we would like to do sometime(tm)

However during intial development everything was centered around MySQL due to the power it offered, and if memory serves, sqlite was still not quite reliable and had several issues.

I can offer you no ETA as towards when sqlite support will be added, if we ever do get around to it. But if someone wishes to submit patches to help make it a reality, they are most welcome :)
BeBot Founder and Fixer Kingpin

Offline Tyrence

  • BeBot User
  • **
  • Posts: 41
  • Karma: +0/-0
Re: SQLite
« Reply #2 on: July 09, 2010, 06:17:12 pm »
It's actually pretty straight forward to do this.

For Budabot, the first thing we did was make sure that MySQL was in standards-compliant mode (there are a few ways to do this; we opted to set an option when the connection is first made).

Then we went through and updated our SQL to make sure it actually was standards-compliant. eg. using GROUP BY to group all columns that weren't part of the agregate, doubling quotes to escape them, etc.

Then lastly we did a few string replacements on incoming queries depending on the database type.  eg. AUTOINCREMENT if it's SQLite, AUTO_INCREMENT if it's MySQL, etc.
« Last Edit: July 09, 2010, 06:19:01 pm by Tyrence »

Offline Glarawyn

  • BeBot Hero
  • ******
  • Posts: 521
  • Karma: +0/-0
Re: SQLite
« Reply #3 on: July 13, 2010, 09:54:41 pm »
SQL Features That SQLite Does Not Implement

  • RIGHT and FULL OUTER JOIN: LEFT OUTER JOIN is implemented, but not RIGHT OUTER JOIN or FULL OUTER JOIN.
  • Complete ALTER TABLE support: Only the RENAME TABLE and ADD COLUMN variants of the ALTER TABLE command are supported. Other kinds of ALTER TABLE operations such as DROP COLUMN, ALTER COLUMN, ADD CONSTRAINT, and so forth are omitted.
  • Complete trigger support: FOR EACH ROW triggers are supported but not FOR EACH STATEMENT triggers.
  • Writing to VIEWs: VIEWs in SQLite are read-only. You may not execute a DELETE, INSERT, or UPDATE statement on a view. But you can create a trigger that fires on an attempt to DELETE, INSERT, or UPDATE a view and do what you need in the body of the trigger.
  • GRANT and REVOKE: Since SQLite reads and writes an ordinary disk file, the only access permissions that can be applied are the normal file access permissions of the underlying operating system. The GRANT and REVOKE commands commonly found on client/server RDBMSes are not implemented because they would be meaningless for an embedded database engine.

The ALTER TABLE one might be painful. Maybe JOINs too. I know Alreadythere uses a number of JOINs in the core modules he wrote. (Flexible Security)

Offline Tyrence

  • BeBot User
  • **
  • Posts: 41
  • Karma: +0/-0
Re: SQLite
« Reply #4 on: July 15, 2010, 06:36:15 pm »
I guess you'd have to determine which of those Bebot is using and then see if there are workarounds.  I know it's not using FULL OUTER JOINs because MySQL doesn't support those either.  RIGHT JOINs can be rewritten as LEFT JOINs so that's not a big deal.

I am not that familliar with Bebot, but I'd be surprised if it was using VIEWs, TRIGGERs, or RIGHT JOINs.

I think the only thing you'd have to handle it ALTER TABLE.  I guess the only option there is to drop and then reload the table.  You could rename the table, create a new table with the correct columns, and then import the data from the old table to the new table if you didn't want to lose the data in there.  That's actually how some RDBMSs handle ALTER TABLE statements.

Offline Glarawyn

  • BeBot Hero
  • ******
  • Posts: 521
  • Karma: +0/-0
Re: SQLite
« Reply #5 on: July 15, 2010, 08:24:18 pm »
Looks like it would be best to disable the ability of multiple bots to use the database when SQLite is in use.

http://sqlite.org/faq.html
Quote
(5) Can multiple applications or multiple instances of the same application access a single database file at the same time?

    Multiple processes can have the same database open at the same time. Multiple processes can be doing a SELECT at the same time. But only one process can be making changes to the database at any moment in time, however.

    SQLite uses reader/writer locks to control access to the database. (Under Win95/98/ME which lacks support for reader/writer locks, a probabilistic simulation is used instead.) But use caution: this locking mechanism might not work correctly if the database file is kept on an NFS filesystem. This is because fcntl() file locking is broken on many NFS implementations. You should avoid putting SQLite database files on NFS if multiple processes might try to access the file at the same time. On Windows, Microsoft's documentation says that locking may not work under FAT filesystems if you are not running the Share.exe daemon. People who have a lot of experience with Windows tell me that file locking of network files is very buggy and is not dependable. If what they say is true, sharing an SQLite database between two or more Windows machines might cause unexpected problems.

    We are aware of no other embedded SQL database engine that supports as much concurrency as SQLite. SQLite allows multiple processes to have the database file open at once, and for multiple processes to read the database at once. When any process wants to write, it must lock the entire database file for the duration of its update. But that normally only takes a few milliseconds. Other processes just wait on the writer to finish then continue about their business. Other embedded SQL database engines typically only allow a single process to connect to the database at once.

    However, client/server database engines (such as PostgreSQL, MySQL, or Oracle) usually support a higher level of concurrency and allow multiple processes to be writing to the same database at the same time. This is possible in a client/server database because there is always a single well-controlled server process available to coordinate access. If your application has a need for a lot of concurrency, then you should consider using a client/server database. But experience suggests that most applications need much less concurrency than their designers imagine.

    When SQLite tries to access a file that is locked by another process, the default behavior is to return SQLITE_BUSY. You can adjust this behavior from C code using the sqlite3_busy_handler() or sqlite3_busy_timeout() API functions.

Offline Tyrence

  • BeBot User
  • **
  • Posts: 41
  • Karma: +0/-0
Re: SQLite
« Reply #6 on: July 15, 2010, 09:54:40 pm »
Looks like it would be best to disable the ability of multiple bots to use the database when SQLite is in use.
It really shouldn't be a problem.

As long as you aren't accessing the db file over the network, and as long as you are running the bot in WinXP or later (or in a *nix environment), it should work.

I know of Budabot setups where linked bots are sharing a sqlite db on Windows, and it's been running like that for several years with no issues [related to the db].

 

* Recent Posts
[AoC] special char for items module by bitnykk
[February 09, 2024, 09:41:18 pm]


0.8.x updates for AoC by bitnykk
[January 30, 2024, 11:16:08 pm]


0.8.x updates for AO by bitnykk
[January 30, 2024, 11:15:37 pm]


BeBot still alive & kicking ! by bitnykk
[December 17, 2023, 12:58:44 am]


Bebot and Rasberry by bitnykk
[November 29, 2023, 11:04:14 pm]

* Who's Online
  • Dot Guests: 674
  • Dot Hidden: 0
  • Dot Users: 0

There aren't any users online.
* Forum Staff
bitnykk admin bitnykk
Administrator
Khalem admin Khalem
Administrator
WeZoN gmod WeZoN
Global Moderator
SimplePortal 2.3.7 © 2008-2024, SimplePortal