app:config:import fails - flag column 'flag_data' is too short ?

[peterjaap]

I got this error; Import failed: Unable to unserialize value, string is corrupted. when running php bin/magento app:config:import to import my app/etc/config.php.

It turns out, the importer does a comparison in \Magento\Config\Model\Config\Importer::import. To do this comparison, it retrieves a 'flag'; the system_config_snapshot flag. This is saved in the database. The problem is; the flag_data column in the flag table in Magento 2 is a TEXT, which is too short; it can only hold 65535 characters! It therefore cuts off the serialized data, making it corrupt.

Maybe this is a solution; ALTER TABLE flag MODIFY flag_data LONGTEXT COMMENT 'Flag Data'; ?

Preconditions

1. Magento 2.2

Steps to reproduce

1. Dump a large enough configuration file and import it (message me for an example file, it contains client data but willing to share with Magento employees) -- you should also be able to create a large config.php by inserting large data sets into the translation table, since it also exports i18n settings.

Expected result

1. It should import app/etc/config.php

Actual result

1. Message; Import failed: Unable to unserialize value, string is corrupted.

[peterjaap]

I circumvented this issue by altering flag_data to a LONGTEXT (from a TEXT) and invalidating the hashes so it will force import my config file; UPDATE flag SET flag_data = '{"system":"","scopes":"","themes":""}' WHERE flag_code = 'config_hash'. Flush your cache after running this query.

Another workaround for me was to remove 'i18n' and 'system' from app/etc/config.php, run UPDATE flag SET flag_data = '{}' WHERE flag_code = 'system_config_snapshot'; and php bin/magento app:config:import.

[peterjaap]

Additional thought; app:config:dump dumps EVERYTHING. Maybe the CLI command needs some extra features to be able to only export it partially (scopes, system, themes, i18n) ?

I'm thinking of leaving out system and i18n by default.

Especially the i18n part can grow very big when somebody has used the inline translation functionality a lot, thereby hitting the MySQL TEXT character limit quite soon.

[peterjaap]

Also, bin/magento setup:upgrade [--keep-generated] runs app:config:import. But when app:config:import fails (see here), setup:upgrade still gives exit code 0. Because it doesn't return a non-zero exit code, my deployment continues even though there is something amiss!

[heldchen]

Additional thought; app:config:dump dumps EVERYTHING.

ideally, there would be a mode that only dumps differences from the current config to the default values.

[peterjaap]

To create a large enough config.php, use https://lipsum.lipsum.com/ to generate a minimum of 66000 bytes of text and place it for example in the default_custom_instructions field in config.php.

Then run php bin/magento app:config:import and run php bin/magento app:config:import again;

Processing configurations data from configuration file...
Import failed: Unable to unserialize value, string is corrupted.

[peterjaap]

Here's a vanilla M2.2 file with a bunch of lipsum; https://gist.github.com/peterjaap/9db722c0c698a1493c61b77dd16af415

[giacmir]

I had this issue on a production website where my serialized config was 71,347 characters long against the 65,535 chars allowed by a TEXT field.

[wejobes]

I also had this issue recently. We are getting ready to deploy a site into production and we have been testing using the pipeline deployment methods as mentioned in the Magento 2.2 docs. To get around the issue, I made a quick, small module that updates the flag_data field to be a MEDIUM_TEXT field. Once in place, and I cleared out the config data in the flag table, everything deployed as expected.

[andy-rush-beerhawk]

I was also having this problem with app:config:import

1. i counted the number of characters in the truncated json before i made the changes: 65534
2. updated the table:
   ALTER TABLE flag MODIFY flag_data LONGTEXT;
   UPDATE flag SET flag_data = '{"system":"","scopes":"","themes":""}' WHERE flag_code = 'config_hash';
   UPDATE flag SET flag_data = '{}' WHERE flag_code = 'system_config_snapshot';
3. Then I ran app:config:import >>> the config import worked and the flag table was updated
4. then I was curious, how large is our config, so i went to count the json again, have a guess how many characters there were? 65534.

I have a feeling this issue may not be fixed with an sql update.