Trouble uploading media

steasdale · June 7, 2022, 6:40pm

Good afternoon

I have been regularly uploading media items to Omeka-S for quite some time with no issues. About one month ago I began receiving the notorious “HTTP ERROR 500” whenever I tried to upload media. I think there is a timeout occurring as I seem to be able to upload very small files (<8MB) but not anything much larger. It doesn’t appear to be a server issue as I can upload the same media outside of Omeka.

The log files contain the following error header when this occurs:

PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

followed by various stack trace items.

I noticed, furthermore, that this problem began right after I was unable to execute two jobs: one multiple items update and one batch export. I had to stop these manually and the Status/Log field in the Jobs page still shows as “Stopping” (not “Stopped”). I have a feeling that this might be causing the problem with the uploading.

Any assistance would be greatly appreciated!

Thanks in advance,
Steven Teasdale

PS. The full log is below:

PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away in /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:112
Stack trace:
#0 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php(112): PDOStatement->execute()
#1 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(1292): Doctrine\DBAL\Driver\PDOStatement->execute()
#2 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/Entity/BasicEntityPersister.php(1825): Doctrine\DBAL\Connection->executeQuery()
#3 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/Entity/BasicEntityPersister.php(1765): Doctrine\ORM\Persisters\Entity\BasicEntityPersister->getOneToManyStatement()
#4 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php(2965): Doctrine\ORM\Persisters\Entity\BasicEntityPersister->loadOneToManyCollection()
#5 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/PersistentCollection.php(729): Doctrine\ORM\UnitOfWork->loadCollection()
#6 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/PersistentCollection.php(228): Doctrine\ORM\PersistentCollection->doInitialize()
#7 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/collections/lib/Doctrine/Common/Collections/AbstractLazyCollection.php(281): Doctrine\ORM\PersistentCollection->initialize()
#8 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Adapter/ItemAdapter.php(267): Doctrine\Common\Collections\AbstractLazyCollection->getIterator()
#9 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php(630): Omeka\Api\Adapter\ItemAdapter->hydrate()
#10 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php(440): Omeka\Api\Adapter\AbstractEntityAdapter->hydrateEntity()
#11 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Manager.php(233): Omeka\Api\Adapter\AbstractEntityAdapter->update()
#12 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Manager.php(136): Omeka\Api\Manager->execute()
#13 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Mvc/Controller/Plugin/Api.php(152): Omeka\Api\Manager->update()
#14 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Controller/Admin/ItemController.php(245): Omeka\Mvc\Controller\Plugin\Api->update()
#15 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/Controller/AbstractActionController.php(77): Omeka\Controller\Admin\ItemController->editAction()
#16 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(321): Laminas\Mvc\Controller\AbstractActionController->onDispatch()
#17 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(178): Laminas\EventManager\EventManager->triggerListeners()
#18 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/Controller/AbstractController.php(103): Laminas\EventManager\EventManager->triggerEventUntil()
#19 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/DispatchListener.php(139): Laminas\Mvc\Controller\AbstractController->dispatch()
#20 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(321): Laminas\Mvc\DispatchListener->onDispatch()
#21 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(178): Laminas\EventManager\EventManager->triggerListeners()
#22 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/Application.php(331): Laminas\EventManager\EventManager->triggerEventUntil()
#23 /home/q1gkdyezwjfd/public_html/omeka-s/index.php(21): Laminas\Mvc\Application->run()
#24 {main}

Next Doctrine\DBAL\Driver\PDO\Exception: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away in /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDO/Exception.php:18
Stack trace:
#0 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php(114): Doctrine\DBAL\Driver\PDO\Exception::new()
#1 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(1292): Doctrine\DBAL\Driver\PDOStatement->execute()
#2 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/Entity/BasicEntityPersister.php(1825): Doctrine\DBAL\Connection->executeQuery()
#3 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/Entity/BasicEntityPersister.php(1765): Doctrine\ORM\Persisters\Entity\BasicEntityPersister->getOneToManyStatement()
#4 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php(2965): Doctrine\ORM\Persisters\Entity\BasicEntityPersister->loadOneToManyCollection()
#5 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/PersistentCollection.php(729): Doctrine\ORM\UnitOfWork->loadCollection()
#6 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/PersistentCollection.php(228): Doctrine\ORM\PersistentCollection->doInitialize()
#7 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/collections/lib/Doctrine/Common/Collections/AbstractLazyCollection.php(281): Doctrine\ORM\PersistentCollection->initialize()
#8 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Adapter/ItemAdapter.php(267): Doctrine\Common\Collections\AbstractLazyCollection->getIterator()
#9 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php(630): Omeka\Api\Adapter\ItemAdapter->hydrate()
#10 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php(440): Omeka\Api\Adapter\AbstractEntityAdapter->hydrateEntity()
#11 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Manager.php(233): Omeka\Api\Adapter\AbstractEntityAdapter->update()
#12 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Manager.php(136): Omeka\Api\Manager->execute()
#13 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Mvc/Controller/Plugin/Api.php(152): Omeka\Api\Manager->update()
#14 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Controller/Admin/ItemController.php(245): Omeka\Mvc\Controller\Plugin\Api->update()
#15 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/Controller/AbstractActionController.php(77): Omeka\Controller\Admin\ItemController->editAction()
#16 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(321): Laminas\Mvc\Controller\AbstractActionController->onDispatch()
#17 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(178): Laminas\EventManager\EventManager->triggerListeners()
#18 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/Controller/AbstractController.php(103): Laminas\EventManager\EventManager->triggerEventUntil()
#19 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/DispatchListener.php(139): Laminas\Mvc\Controller\AbstractController->dispatch()
#20 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(321): Laminas\Mvc\DispatchListener->onDispatch()
#21 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(178): Laminas\EventManager\EventManager->triggerListeners()
#22 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/Application.php(331): Laminas\EventManager\EventManager->triggerEventUntil()
#23 /home/q1gkdyezwjfd/public_html/omeka-s/index.php(21): Laminas\Mvc\Application->run()
#24 {main}

Next Doctrine\DBAL\Exception\ConnectionLost: An exception occurred while executing ‘SELECT t1.id AS id_2, t1.title AS title_3, t1.is_public AS is_public_4, t1.created AS created_5, t1.modified AS modified_6, t0.ingester AS ingester_7, t0.renderer AS renderer_8, t0.data AS data_9, t0.source AS source_10, t0.media_type AS media_type_11, t0.storage_id AS storage_id_12, t0.extension AS extension_13, t0.sha256 AS sha256_14, t0.size AS size_15, t0.has_original AS has_original_16, t0.has_thumbnails AS has_thumbnails_17, t0.position AS position_18, t0.lang AS lang_19, t0.alt_text AS alt_text_20, t1.owner_id AS owner_id_21, t1.resource_class_id AS resource_class_id_22, t1.resource_template_id AS resource_template_id_23, t1.thumbnail_id AS thumbnail_id_24, t0.item_id AS item_id_25, t1.resource_type FROM media t0 INNER JOIN resource t1 ON t0.id = t1.id WHERE t0.item_id = ? ORDER BY t0.position ASC’ with params [36573]:

SQLSTATE[HY000]: General error: 2006 MySQL server has gone away in /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php:115
Stack trace:
#0 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php(182): Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException()
#1 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php(159): Doctrine\DBAL\DBALException::wrapException()
#2 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(2214): Doctrine\DBAL\DBALException::driverExceptionDuringQuery()
#3 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(1301): Doctrine\DBAL\Connection->handleExceptionDuringQuery()
#4 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/Entity/BasicEntityPersister.php(1825): Doctrine\DBAL\Connection->executeQuery()
#5 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/Entity/BasicEntityPersister.php(1765): Doctrine\ORM\Persisters\Entity\BasicEntityPersister->getOneToManyStatement()
#6 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php(2965): Doctrine\ORM\Persisters\Entity\BasicEntityPersister->loadOneToManyCollection()
#7 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/PersistentCollection.php(729): Doctrine\ORM\UnitOfWork->loadCollection()
#8 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/PersistentCollection.php(228): Doctrine\ORM\PersistentCollection->doInitialize()
#9 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/doctrine/collections/lib/Doctrine/Common/Collections/AbstractLazyCollection.php(281): Doctrine\ORM\PersistentCollection->initialize()
#10 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Adapter/ItemAdapter.php(267): Doctrine\Common\Collections\AbstractLazyCollection->getIterator()
#11 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php(630): Omeka\Api\Adapter\ItemAdapter->hydrate()
#12 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php(440): Omeka\Api\Adapter\AbstractEntityAdapter->hydrateEntity()
#13 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Manager.php(233): Omeka\Api\Adapter\AbstractEntityAdapter->update()
#14 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Api/Manager.php(136): Omeka\Api\Manager->execute()
#15 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Mvc/Controller/Plugin/Api.php(152): Omeka\Api\Manager->update()
#16 /home/q1gkdyezwjfd/public_html/omeka-s/application/src/Controller/Admin/ItemController.php(245): Omeka\Mvc\Controller\Plugin\Api->update()
#17 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/Controller/AbstractActionController.php(77): Omeka\Controller\Admin\ItemController->editAction()
#18 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(321): Laminas\Mvc\Controller\AbstractActionController->onDispatch()
#19 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(178): Laminas\EventManager\EventManager->triggerListeners()
#20 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/Controller/AbstractController.php(103): Laminas\EventManager\EventManager->triggerEventUntil()
#21 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/DispatchListener.php(139): Laminas\Mvc\Controller\AbstractController->dispatch()
#22 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(321): Laminas\Mvc\DispatchListener->onDispatch()
#23 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-eventmanager/src/EventManager.php(178): Laminas\EventManager\EventManager->triggerListeners()
#24 /home/q1gkdyezwjfd/public_html/omeka-s/vendor/laminas/laminas-mvc/src/Application.php(331): Laminas\EventManager\EventManager->triggerEventUntil()
#25 /home/q1gkdyezwjfd/public_html/omeka-s/index.php(21): Laminas\Mvc\Application->run()
#26 {main}

jflatnes · June 7, 2022, 6:53pm

How much time would you say it’s taking to try an upload before you see that error?

The “MySQL has gone away” error basically means that we can’t contact the database. One reason that can happen is indeed a timeout: there’s a setting on the MySQL server side for how long to wait for a connection to send a command, after which time the server hangs up and you get that kind of error. This would probably be after a pretty noticeably long time waiting, though. The MySQL server setting that controls this is called wait_timeout.

What kinds of files are these that you’re trying to upload, images? Something else? And, what kinds of modules do you have installed? Anything that has to process uploaded files and could be contributing to it taking a long time?

steasdale · June 7, 2022, 8:23pm

It tries to upload the media item for about 75 seconds before the error occurs. Before the problem started to occur it would typically take about 3 to 4 seconds to upload a typical media item.

We are running Omeka-S through shared hosting (shared business server on GoDaddy) and so cannot make changes to the MySQL wait_timeout setting.

The media items are JPEG images, ranging from 3 to 5 MB in size.

We have a number of installed modules, listed below. I don’t think any of these modules needs to process imported media files.

Block Plus
Blocks Disposition
Bulk Edit
Bulk Export
CSS Editor
CSV Import
Custom Vocab
Data Type RDF
Easy Install
Generic Module
ItemCopy
Log
Mapping
Numeric Data Types
PDF Embed
Scripto
Value Suggest
Zotero Import

Thanks again,
Steven

jflatnes · June 8, 2022, 8:08pm

75 seconds is definitely way too long for this to be taking.

Try this for me: you can set some options in your installation’s config/local.config.php file that affect how thumbnails are made. Let’s see if we can narrow down what seems to be causing all this delay.

The configuration is explained in the manual, and in particular here I’d like you to look at changing the thumbnailer (the “service_manager” “aliases” part described on that manual link).

For a quick check, you could switch it to “NoThumbnail” (this is mentioned at the end of the manual section I linked to), which will just disable automatic thumbnails. If your uploads are fast again after doing that, then it’s the thumbnailing process that’s taking all that extra time. If they’re still slow, it’s something else. Either way, it would be useful information for figuring out what’s going on here.

steasdale · June 9, 2022, 5:16pm

OK, I set the thumbnailer to NoThumbnail and the upload speed is back to normal… the JPG files now upload immediately.

jflatnes · June 9, 2022, 5:25pm

OK. Well, that certainly helps pinpoint the issue.

Can you share an example file that’s causing you trouble? Maybe we can tell something from that.

If ImageMagick is the problem here, you might have to just check with your host on that front. Really all we do is run the command-line ImageMagick on the files that are uploaded. If there’s an issue with ImageMagick being very slow, it’s somewhat out of our hands.

You do have some other options you could try on your own, which are mentioned in that same manual section I linked you to before: the Gd and Imagick thumbnailers. The GD extension is pretty much always available and supports JPEG, though you could hit memory limit problems depending on the server configuration with larger-dimension images (which these probably are).

If your server has the imagick extension installed (you can check the “System information” screen linked on the bottom of each admin page to see if it’s listed), then the Imagick thumbnailer is also an option. It also just uses ImageMagick, but it’s possible that using it would resolve this problem.

steasdale · June 9, 2022, 5:53pm

Here is an example file. The server does have the Imagick extension installed so I will give that a try.

steasdale · June 9, 2022, 6:03pm

A quick update.

Everything works fine with the Gd thumbnailer. Both the ImageMagick and Imagick thumbnailers no longer work on most images, for some reason, although the extensions are installed on the server.

I will continue using GD for now. Thanks again for all of your help.

jflatnes · June 9, 2022, 6:19pm

Good to hear that GD is working.

You might talk to your host and give them the example file also to see if they can reproduce and maybe do something about the slowness.

I had no issues with the file you shared, so I suspect the issue is somewhat unique to your host/server.

system · February 14, 2023, 6:20pm

This topic was automatically closed 250 days after the last reply. New replies are no longer allowed.