Strange error in edit form by saving the field with utf8 content

[rrit]

Hello @johnny,

System data:

• PHP8.1, GC 3.0.9, CI 4.3.7
• Table field varchar(255), table charset utf8mb3

GC DB Adapter configured with

                'driver' => 'mysqli', // (with Pdo_Mysql exactly the same behaviour)
...
                'charset'  => 'utf8',

Field contain the text (copied 1:1 directly from database):

<p>Dirk van Gunsteren, 1953 geboren, übersetzte u.a. Jonathan Safran Foer, Colum McCann, Thomas Pynchon, Philip Roth, T.C. Boyle und Oliver Sacks. 2007 erhielt er den Heinrich Maria Ledig-Rowohlt-Preis.</p>
<p>Oliver Sacks, geboren 1933 in London, war Pro

(the text is exactly 255 symbols long, you can check it)

This content will be correct presented if record called to view and will be correct added in form field if record called for edit, but if you want to save this record from edit it produce CI error, hier is the trace:

CRITICAL - 2023-08-10 15:01:42 --> SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'redaktion' at row 1
in VENDORPATH/laminas/laminas-db/src/Adapter/Driver/Pdo/Statement.php on line 212.
 1 VENDORPATH/laminas/laminas-db/src/Adapter/Driver/Pdo/Statement.php(212): PDOStatement->execute()
 2 VENDORPATH/grocery-crud/enterprise/src/GroceryCrud/Core/Model.php(563): Laminas\Db\Adapter\Driver\Pdo\Statement->execute()
 3 VENDORPATH/grocery-crud/enterprise/src/GroceryCrud/Core/State/UpdateState.php(107): GroceryCrud\Core\Model->update()
 4 VENDORPATH/grocery-crud/enterprise/src/GroceryCrud/Core/State/StateAbstract.php(1177): GroceryCrud\Core\State\UpdateState->GroceryCrud\Core\State\{closure}()
 5 VENDORPATH/grocery-crud/enterprise/src/GroceryCrud/Core/State/StateAbstract.php(331): GroceryCrud\Core\State\StateAbstract->stateOperation()
 6 VENDORPATH/grocery-crud/enterprise/src/GroceryCrud/Core/State/UpdateState.php(118): GroceryCrud\Core\State\StateAbstract->stateOperationWithCallbacks()
 7 VENDORPATH/grocery-crud/enterprise/src/GroceryCrud/Core/GroceryCrud.php(2867): GroceryCrud\Core\State\UpdateState->render()
 8 APPPATH/Controllers/Products.php(296): GroceryCrud\Core\GroceryCrud->render()
 9 SYSTEMPATH/CodeIgniter.php(932): App\Controllers\Products->manage()
10 SYSTEMPATH/CodeIgniter.php(497): CodeIgniter\CodeIgniter->runController()
11 SYSTEMPATH/CodeIgniter.php(366): CodeIgniter\CodeIgniter->handleRequest()
12 FCPATH/index.php(67): CodeIgniter\CodeIgniter->run()

I checked the codepage in browser, is utf8 too.

This record was initially saved to db with CI active record and already truncated right to 255 utf8 symbols.

Are laminas-db adapter treat the utf8 content differently of CI-adapter? Can it be tuned anywhere with config options?

IMPORTANT:
same system, but with PHP7.4 - no error!

Thank you!

[johnny]

Hello @rrit,

That’s a strange error that it is the first time that I see it to be honest. I’ve checked your text and it seems that it is probably 256 chars and not 255. So the text:

<p>Dirk van Gunsteren, 1953 geboren, übersetzte u.a. Jonathan Safran Foer, Colum McCann, Thomas Pynchon, Philip Roth, T.C. Boyle und Oliver Sacks. 2007 erhielt er den Heinrich Maria Ledig-Rowohlt-Preis.</p>
><p>Oliver Sacks, geboren 1933 in London, war Pro

Is most possibly translated to:

<p>Dirk van Gunsteren, 1953 geboren, übersetzte u.a. Jonathan Safran Foer, Colum McCann, Thomas Pynchon, Philip Roth, T.C. Boyle und Oliver Sacks. 2007 erhielt er den Heinrich Maria Ledig-Rowohlt-Preis.</p>\n<p>Oliver Sacks, geboren 1933 in London, war Pro

with “\n” instead of a new line. I am not sure if this is a laminas-db issue or a MYSQL configuration issue (e.g. for mySQL to throw an error instead of just cut the string) but for now, can I suggest having a rule for that which is more correct anyway (e.g. you will not rely on the MySQL/PHP configuration or version).

More specifically on purpose I have a frontend limitation on the input text. But for textarea I don’t because usually the text is type “TEXT”. So I would suggest having something like this (documentation for valitron here: GitHub - vlucas/valitron: Valitron is a simple, elegant, stand-alone validation library with NO dependencies ):

$crud->setRule('customer_name', 'lengthMax', 255);

To make sure that the length of the text will not exceed the number that you have within the database.

Let me know if that worked.

Regards
Johnny

[rrit]

Hello @johnny,

your right, the string consist “\n”, but from the database point of view the content is 255 symbols long (0x00-0xFF), see screenshot here:

And Codeigniter’s Active Record agreed with it too and show no warning or error.

With

$crud->setRule('redaktion', 'lengthMax', 255);

I become indeed a warning about the content length, but more curious is further, pls see here:

Screencast

After that the same length in DB but no errors.

“\n” between </p><p> was eliminated but “2” at the end added:

???

[johnny]

That’s weird. First of all, I have no idea what the 2 at the end is . And second, maybe we are looking at the wrong character? Maybe the symbol ü is the problem? Can you check if possible?

[rrit]

I have added the “2” myself in the field.
I mean really added without to remove any symbols before.
Have you seen the screencast ?

Then with click on save the “\n” (0x0A) will be magically deleted (or already before the click, just by adding “2” at the end of text in field?) and content with added “2” will be saved into db as string 255 characters long.

May be it can be explained with different treatment of unicode multi-byte symbols (f.ex. here “ü” 0xC3/0xBC/ as you say) by CI- and laminas-db adapters, or even by frontend (GC form / JS)…

And again: same setup with PHP7.4 - no errors!

[johnny]

I have just seen the screencast. Ok, now this is weird indeed . Probably something with the utf-8 encoding but nothing that I can see from a quick google search that I did for lamina-db or PHP 8. I will keep this in mind to see if I find something in the future.