CSV imports stalling

Starting yesterday every CSV import I try stalls while the import is still in the queued stage.

  • Omeka 2.5
  • CSV Import 2.0.4
  • PHP 5.6.40 (cgi-fcgi)
  • MySQL Server 5.6.41
  • Imports worked just fine a week or two ago
  • I’ve tried files that used to import fine and simple test files. Everything fails.
  • No error is displayed on screen, but errors do appear in the log at omeka/admin/error_log (copied below)
  • We have made no changes on our end (no new plugins or updates or settings changes) and our web host (Bluehost) says they have made no changes on their end (other than the few experimental changes they made while trying to diagnose this problem).

Any insights would be appreciated! Clearly something has changed, but I’m not sure what. Here are the errors that appear in the log after an attempted import:

[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'mssql.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mssql.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mssql.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mssql.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mssql.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqlnd_mysql.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysql.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysql.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysql.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysql.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqlnd_mysqli.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysqli.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysqli.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysqli.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysqli.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_dblib.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_dblib.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_dblib.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_dblib.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_dblib.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_mysqlnd.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_mysqlnd.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_mysqlnd.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_mysqlnd.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_mysqlnd.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'ixed.5.6.lin' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/ixed.5.6.lin (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/ixed.5.6.lin: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/ixed.5.6.lin.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/ixed.5.6.lin.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'mssql.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mssql.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mssql.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mssql.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mssql.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqlnd_mysql.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysql.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysql.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysql.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysql.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqlnd_mysqli.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysqli.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysqli.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysqli.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/mysqlnd_mysqli.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_dblib.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_dblib.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_dblib.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_dblib.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_dblib.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_mysqlnd.so' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_mysqlnd.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_mysqlnd.so: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_mysqlnd.so.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/pdo_mysqlnd.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'ixed.5.6.lin' (tried: /opt/cpanel/ea-php72/root/usr/lib64/php/modules/ixed.5.6.lin (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/ixed.5.6.lin: cannot open shared object file: No such file or directory), /opt/cpanel/ea-php72/root/usr/lib64/php/modules/ixed.5.6.lin.so (/opt/cpanel/ea-php72/root/usr/lib64/php/modules/ixed.5.6.lin.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
[24-Apr-2019 16:04:41 UTC] PHP Fatal error:  Uncaught Zend_Db_Adapter_Mysqli_Exception: The Mysqli extension is required for this adapter but the extension is not loaded in /home3/forbesli/public_html/omeka/application/libraries/Zend/Db/Adapter/Mysqli.php:291
Stack trace:
#0 /home3/forbesli/public_html/omeka/application/libraries/Zend/Db/Adapter/Abstract.php(460): Zend_Db_Adapter_Mysqli->_connect()
#1 /home3/forbesli/public_html/omeka/application/libraries/Zend/Db/Adapter/Abstract.php(811): Zend_Db_Adapter_Abstract->query('SELECT name, va...', Array)
#2 /home3/forbesli/public_html/omeka/application/libraries/Omeka/Db.php(79): Zend_Db_Adapter_Abstract->fetchPairs('SELECT name, va...')
#3 /home3/forbesli/public_html/omeka/application/libraries/Omeka/Application/Resource/Options.php(33): Omeka_Db->__call('fetchPairs', Array)
#4 /home3/forbesli/public_html/omeka/application/libraries/Zend/Application/Bootstrap/BootstrapAbstract.php(695): Omeka_Application_Resource_Options->init()
#5 /home3/forbesli/public_html/omeka/application/libraries/Zend/Application/Bo in /home3/forbesli/public_html/omeka/application/libraries/Zend/Db/Adapter/Mysqli.php on line 291

This looks like a server problem with your PHP installation.

The PHP CLI binary looks like it’s trying to load extensions from the wrong location (the path refers to PHP 7.2 even though you say you’re using 5.6). This is probably something you’d have to talk with your host about.

Thanks, @jflatnes. Bluehost swears they have made no changes and that it is our code (i.e. Omeka or one of it’s plugins) that must be requesting the PHP 7.2 paths. This brings up a number of questions:

  • Does what Bluehost is telling me make sense? I find it surprising, but I’ve gotten a very consistent message from a number of their reps.
  • Is there a log where I can see what updates have been run on Omeka plugins lately, or some other way of checking this information? (So that I could roll back the offending changes.)
  • Is there any other way you can imagine us resolving this problem.

It may be relevant to know that Omeka lists the following PHP extensions under System Information. Interestingly, this includes some of the libraries that aren’t being loaded according to the log, which makes me think that the processes spawned by CSV Import may be looking for libraries in a different place from the rest of Omeka.

PHP Extensions

Regular bcmath, bz2, calendar, cgi-fcgi, Core, ctype, curl, date, dom, ereg, exif, fileinfo, filter, ftp, gd, gettext, gmp, hash, iconv, imagick, imap, intl, ionCube Loader, json, libxml, mbstring, mcrypt, mhash, mssql, mysql, mysqli, mysqlnd, OAuth, odbc, openssl, pcntl, pcre, PDO, pdo_dblib, pdo_mysql, PDO_ODBC, pdo_pgsql, pdo_sqlite, pgsql, Phar, posix, pspell, readline, Reflection, session, SimpleXML, soap, sockets, SourceGuardian, SPL, sqlite3, standard, tidy, tokenizer, wddx, xml, xmlreader, xmlrpc, xmlwriter, xsl, Zend Guard Loader, zip, zlib
Zend the ionCube PHP Loader (enabled) + Intrusion Protection from ioncube24.com (unconfigured), Zend Guard Loader

I’m pretty confident in saying this isn’t coming from the Omeka end of things…

Omeka just runs the php binary to run background jobs: some problem with the binary and/or the php.ini it uses is causing it to try to load extensions from the wrong place.

Omeka really only has one relevant setting: background.php.path in application/config/config.ini, where you can set the path to the PHP command on your server (if it’s not set, Omeka just looks for it, effectively running which php).

Thanks, John. When I SSH in to the server and run PHP --ini I can see that all the configuration files live below /opt/cpanel/ea-php72/. Furthermore, running PHP --version reveals that 7.2.17 is being run. This is true of all paths for PHP which are revealed when I run which --all php.

It sounds background.php.path may be part of the solution, but I would need to find a path to the correct version of PHP first, correct? Any insight you could provide into this would be much appreciated. I’ve found our web host to be decidedly uncooperative on this matter.

Thanks!

Yeah i mean it’s certainly suspicious that you’re running PHP 5.6 on the server but 7.2 on the command line…

Are there other folders under /opt/cpanel which might correspond with other versions of PHP? In particular something referencing 5.6?

It seems like maybe Cpanel or something like it is setting the php.ini up for your web configuration correctly, so referring to PHP 5.6 extensions and things that are installed, but that same configuration is being used for the totally different command-line version. If so, one option would be, as you say, to find the correct php binary and just point background.php.path to it.

Really whatever server configuration thing that’s in use here should be properly handling both the CLI and CGI versions, but it looks like that’s not happening for whatever reason.

Yes, I was able to find an executable for 5.6.40 at /opt/cpanel/ea-php56/root/usr/bin/php Do you recommend I set

background.php.path = "/opt/cpanel/ea-php56/root/usr/bin/php"

in application/config/config.ini?

Yes, that looks promising.

With the change to config.ini everything seems to be working once more. Thank you, John!