Omeka API Import Error

I’m migrating a classic Omeka install with about 2500 items to Omeka S, using the Omeka 2 Importer module in Omeka S.

The API module appears to work, but is generating some errors in the log, and then it fails every time at 1800 items.

Can someone help me decode the log file? It seems like there must be one particular item that it can’t process and that brings the import to a halt.

Any suggestions would be appreciated. I have direct access to the old classic install. I do not have direct server access to the Omeka S destination, only access through the Omeka interface.
Thanks!
Dan

[long log file:]

2019-10-24T15:54:57+00:00 INFO (6): Importing item page 1
2019-10-24T15:56:34+00:00 INFO (6): Importing item page 2
2019-10-24T15:57:02+00:00 INFO (6): Importing item page 3
2019-10-24T15:57:14+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omeka3l6SSe[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaCi98pW.jpg’” failed with status code 1.
2019-10-24T15:57:19+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omeka0kNanE[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaQnisdA.jpg’” failed with status code 1.
2019-10-24T15:57:26+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaUvypew[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omeka9tFFuJ.jpg’” failed with status code 1.
2019-10-24T15:57:28+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaWZxs0W[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekads3fgg.jpg’” failed with status code 1.
2019-10-24T15:57:34+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekacTLqCz[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekajMNmJ8.jpg’” failed with status code 1.
2019-10-24T15:57:39+00:00 ERR (3): Command “/usr/bin/convert -density 150 ‘/tmp/omekabzJTwB[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaocCtCn.jpg’” failed with status code 1.
2019-10-24T15:57:39+00:00 ERR (3): Command “/usr/bin/convert -density 150 ‘/tmp/omekaks2809[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekatZQRtW.jpg’” failed with status code 1.
2019-10-24T15:58:14+00:00 INFO (6): Importing item page 4
2019-10-24T15:59:33+00:00 ERR (3): Command “/usr/bin/convert -density 150 ‘/tmp/omekacm7IGp[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekabRdLgZ.jpg’” failed with status code 1.
2019-10-24T15:59:40+00:00 INFO (6): Importing item page 5
2019-10-24T16:00:20+00:00 INFO (6): Importing item page 6
2019-10-24T16:01:52+00:00 INFO (6): Importing item page 7
2019-10-24T16:02:24+00:00 INFO (6): Importing item page 8
2019-10-24T16:02:50+00:00 INFO (6): Importing item page 9
2019-10-24T16:03:13+00:00 INFO (6): Importing item page 10
2019-10-24T16:03:37+00:00 INFO (6): Importing item page 11
2019-10-24T16:04:36+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaKZgjtb[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaE0bOYz.jpg’” failed with status code 1.
2019-10-24T16:04:37+00:00 INFO (6): Importing item page 12
2019-10-24T16:06:08+00:00 INFO (6): Importing item page 13
2019-10-24T16:06:39+00:00 INFO (6): Importing item page 14
2019-10-24T16:08:03+00:00 INFO (6): Importing item page 15
2019-10-24T16:09:22+00:00 INFO (6): Importing item page 16
2019-10-24T16:10:21+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaKZWEUt[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaGBkBWs.jpg’” failed with status code 1.
2019-10-24T16:10:31+00:00 ERR (3): Command “/usr/bin/convert -density 150 ‘/tmp/omekaI8z8S2[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaYMLShq.jpg’” failed with status code 1.
2019-10-24T16:10:36+00:00 INFO (6): Importing item page 17
2019-10-24T16:11:41+00:00 INFO (6): Importing item page 18
2019-10-24T16:12:57+00:00 INFO (6): Importing item page 19
2019-10-24T16:13:48+00:00 INFO (6): Importing item page 20
2019-10-24T16:14:42+00:00 INFO (6): Importing item page 21
2019-10-24T16:15:02+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaCDofGD[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omeka5f70pu.jpg’” failed with status code 1.
2019-10-24T16:15:37+00:00 INFO (6): Importing item page 22
2019-10-24T16:16:47+00:00 INFO (6): Importing item page 23
2019-10-24T16:17:44+00:00 INFO (6): Importing item page 24
2019-10-24T16:18:18+00:00 INFO (6): Importing item page 25
2019-10-24T16:19:29+00:00 INFO (6): Importing item page 26
2019-10-24T16:20:34+00:00 INFO (6): Importing item page 27
2019-10-24T16:21:50+00:00 INFO (6): Importing item page 28
2019-10-24T16:22:46+00:00 INFO (6): Importing item page 29
2019-10-24T16:24:13+00:00 INFO (6): Importing item page 30
2019-10-24T16:25:23+00:00 INFO (6): Importing item page 31
2019-10-24T16:26:51+00:00 INFO (6): Importing item page 32
2019-10-24T16:28:07+00:00 INFO (6): Importing item page 33
2019-10-24T16:29:36+00:00 INFO (6): Importing item page 34
2019-10-24T16:30:45+00:00 INFO (6): Importing item page 35
2019-10-24T16:32:55+00:00 INFO (6): Importing item page 36
2019-10-24T16:33:31+00:00 ERR (3): Command “/usr/bin/convert -density 150 ‘/tmp/omekagv0XfO[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omeka69K0Qw.jpg’” failed with status code 1.
2019-10-24T16:33:42+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaoJNS0D[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekapwlqUN.jpg’” failed with status code 1.
2019-10-24T16:33:42+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaWAJrVX[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekayqdxV8.jpg’” failed with status code 1.
2019-10-24T16:33:43+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekabz1J0j[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaGNM5wx.jpg’” failed with status code 1.
2019-10-24T16:33:44+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekap6QR8K[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekamiWU7Z.jpg’” failed with status code 1.
2019-10-24T16:33:45+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaUZsXbf[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaHCh4Gw.jpg’” failed with status code 1.
2019-10-24T16:33:45+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaDQb3hO[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaHANSc7.jpg’” failed with status code 1.
2019-10-24T16:33:46+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekacaw3cq[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaQ4VaiK.jpg’” failed with status code 1.
2019-10-24T16:33:46+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekawCZjs4[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekadJP9tp.jpg’” failed with status code 1.
2019-10-24T16:33:47+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaoE6rCK[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaFMRu26.jpg’” failed with status code 1.
2019-10-24T16:33:47+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaK3EpBt[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekahieAVR.jpg’” failed with status code 1.
2019-10-24T16:33:48+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekawJ0mmg[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaKIMpYF.jpg’” failed with status code 1.
2019-10-24T16:33:48+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaXj4JG5[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekawyMigw.jpg’” failed with status code 1.
2019-10-24T16:33:49+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omeka1pE0UW[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekadeoRno.jpg’” failed with status code 1.
2019-10-24T16:33:49+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekaLvLqZP[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaQbDWbj.jpg’” failed with status code 1.
2019-10-24T16:33:49+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekanhBpuM[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekai36feg.jpg’” failed with status code 1.
2019-10-24T16:33:50+00:00 ERR (3): Command “/usr/bin/convert ‘/tmp/omekafrC71J[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaz5K05f.jpg’” failed with status code 1.
2019-10-24T16:34:03+00:00 ERR (3): Command “/usr/bin/convert -density 150 ‘/tmp/omekaIZKEWO[0]’ -auto-orient -background white +repage -alpha remove -thumbnail ‘800x800>’ ‘/tmp/omekaPNPfcR.jpg’” failed with status code 1.
2019-10-24T16:34:31+00:00 INFO (6): Importing item page 37
2019-10-24T16:34:37+00:00 ERR (3): exception ‘Omeka\Api\Exception\ValidationException’ in /var/www/html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php:563
Errors:
{
“o:media”: [
{
“file”: [
“Error validating “/dm2015/files/original/5a97a7ae0f60eb73436d416e6bb82adb.docx”. Cannot store files with the media type “application/vnd.openxmlformats-officedocument.wordprocessingml.document”.”
]
}
]
}
Stack trace:
#0 /var/www/html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php(274): Omeka\Api\Adapter\AbstractEntityAdapter->hydrateEntity(Object(Omeka\Api\Request), Object(Omeka\Entity\Item), Object(Omeka\Stdlib\ErrorStore))
#1 /var/www/html/omeka-s/application/src/Api/Manager.php(224): Omeka\Api\Adapter\AbstractEntityAdapter->create(Object(Omeka\Api\Request))
#2 /var/www/html/omeka-s/application/src/Api/Manager.php(78): Omeka\Api\Manager->execute(Object(Omeka\Api\Request))
#3 /var/www/html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php(317): Omeka\Api\Manager->create(‘items’, Array, Array, Array)
#4 /var/www/html/omeka-s/application/src/Api/Manager.php(227): Omeka\Api\Adapter\AbstractEntityAdapter->batchCreate(Object(Omeka\Api\Request))
#5 /var/www/html/omeka-s/application/src/Api/Manager.php(97): Omeka\Api\Manager->execute(Object(Omeka\Api\Request))
#6 /var/www/html/omeka-s/modules/Omeka2Importer/src/Job/Import.php(226): Omeka\Api\Manager->batchCreate(‘items’, Array, Array, Array)
#7 /var/www/html/omeka-s/modules/Omeka2Importer/src/Job/Import.php(202): Omeka2Importer\Job\Import->createItems(Array)
#8 /var/www/html/omeka-s/modules/Omeka2Importer/src/Job/Import.php(83): Omeka2Importer\Job\Import->importItems(Array)
#9 /var/www/html/omeka-s/application/src/Job/DispatchStrategy/Synchronous.php(34): Omeka2Importer\Job\Import->perform()
#10 /var/www/html/omeka-s/application/src/Job/Dispatcher.php(105): Omeka\Job\DispatchStrategy\Synchronous->send(Object(Omeka\Entity\Job))
#11 /var/www/html/omeka-s/application/data/scripts/perform-job.php(43): Omeka\Job\Dispatcher->send(Object(Omeka\Entity\Job), Object(Omeka\Job\DispatchStrategy\Synchronous))
#12 {main}
2019-10-24T16:35:03+00:00 ERR (3): Doctrine\ORM\ORMInvalidArgumentException: A new entity was found through the relationship ‘Mapping\Entity\MappingMarker#item’ that was not configured to cascade persist operations for entity: Omeka\Entity\Item@000000000acaea2b000000005ac94b3a. To solve this issue: Either explicitly call EntityManager#persist() on this unknown entity or configure cascade persist this association in the mapping for example @ManyToOne(…,cascade={“persist”}). If you cannot find out which entity causes the problem implement ‘Omeka\Entity\Item#__toString()’ to get a clue. in /var/www/html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/ORMInvalidArgumentException.php:92
Stack trace:
#0 /var/www/html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php(852): Doctrine\ORM\ORMInvalidArgumentException::newEntityFoundThroughRelationship(Array, Object(Omeka\Entity\Item))
#1 /var/www/html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php(740): Doctrine\ORM\UnitOfWork->computeAssociationChanges(Array, Object(Omeka\Entity\Item))
#2 /var/www/html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php(452): Doctrine\ORM\UnitOfWork->computeChangeSet(Object(Doctrine\ORM\Mapping\ClassMetadata), Object(Mapping\Entity\MappingMarker))
#3 /var/www/html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php(765): Doctrine\ORM\UnitOfWork->computeScheduleInsertsChangeSets()
#4 /var/www/html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php(340): Doctrine\ORM\UnitOfWork->computeChangeSets()
#5 /var/www/html/omeka-s/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php(356): Doctrine\ORM\UnitOfWork->commit(NULL)
#6 /var/www/html/omeka-s/application/src/Api/Adapter/AbstractEntityAdapter.php(332): Doctrine\ORM\EntityManager->flush()
#7 /var/www/html/omeka-s/application/src/Api/Manager.php(227): Omeka\Api\Adapter\AbstractEntityAdapter->batchCreate(Object(Omeka\Api\Request))
#8 /var/www/html/omeka-s/application/src/Api/Manager.php(97): Omeka\Api\Manager->execute(Object(Omeka\Api\Request))
#9 /var/www/html/omeka-s/modules/Omeka2Importer/src/Job/Import.php(226): Omeka\Api\Manager->batchCreate(‘items’, Array, Array, Array)
#10 /var/www/html/omeka-s/modules/Omeka2Importer/src/Job/Import.php(202): Omeka2Importer\Job\Import->createItems(Array)
#11 /var/www/html/omeka-s/modules/Omeka2Importer/src/Job/Import.php(83): Omeka2Importer\Job\Import->importItems(Array)
#12 /var/www/html/omeka-s/application/src/Job/DispatchStrategy/Synchronous.php(34): Omeka2Importer\Job\Import->perform()
#13 /var/www/html/omeka-s/application/src/Job/Dispatcher.php(105): Omeka\Job\DispatchStrategy\Synchronous->send(Object(Omeka\Entity\Job))
#14 /var/www/html/omeka-s/application/data/scripts/perform-job.php(43): Omeka\Job\Dispatcher->send(Object(Omeka\Entity\Job), Object(Omeka\Job\DispatchStrategy\Synchronous))
#15 {main}

What versions of Omeka S, the Omeka 2 Importer, and the Mapping module are you using?

One easy problem to see is that it’s trying to import a .docx file and failing: you need to add application/vnd.openxmlformats-officedocument.wordprocessingml.document to your list of allowed mime types in the Omeka S security settings.

The final error may or may not be related.

Thanks for the quick reply!
We are running Omeka S version 1.4.0 (details follow)
And Omeka 2 Importer 1.1.0
Mapping version 1.0.2

On the classic side the install is Omeka 2.3.1
I’m really hesitant to upgrade it if we can just get the info to transfer over the api instead.

I will look to see if I can find the docx file in on the source platform. If there are only one or two they could be moved manually, especially if that’s what causes the upload to stall after 1800 items.

Dan

details of Omeka S follow:

Omeka S
Version 1.4.0
PHP
Version 7.2.19-0ubuntu0.18.04.2
SAPI apache2handler
Memory Limit 128M
POST Size Limit 8M
File Upload Limit 2M
Garbage Collection Yes
Extensions apache2handler, calendar, Core, ctype, date, dom, exif, fileinfo, filter, ftp, gettext, hash, iconv, json, libxml, mysqli, mysqlnd, openssl, pcre, PDO, pdo_mysql, Phar, posix, readline, Reflection, session, shmop, SimpleXML, sockets, sodium, SPL, standard, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xml, xmlreader, xmlwriter, xsl, Zend OPcache, zlib
Disabled Functions , pcntl_alarm, pcntl_async_signals, pcntl_exec, pcntl_fork, pcntl_getpriority, pcntl_get_last_error, pcntl_setpriority, pcntl_signal, pcntl_signal_dispatch, pcntl_signal_get_handler, pcntl_sigprocmask, pcntl_sigtimedwait, pcntl_sigwaitinfo, pcntl_strerror, pcntl_wait, pcntl_waitpid, pcntl_wexitstatus, pcntl_wifcontinued, pcntl_wifexited, pcntl_wifsignaled, pcntl_wifstopped, pcntl_wstopsig, pcntl_wtermsig
MySQL
Server Version 5.7.27-0ubuntu0.18.04.1
Client Version mysqlnd 5.0.12-dev - 20150407 - $Id: 3591daad22de08524295e1bd073aceeff11e6579 $
Mode ONLY_FULL_GROUP_BY, STRICT_TRANS_TABLES, NO_ZERO_IN_DATE, NO_ZERO_DATE, ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USER, NO_ENGINE_SUBSTITUTION
OS
Version Linux 4.15.0-58-generic x86_64

Adding that type to the allowed list is just a simple change to the settings page of the Omeka S install; I’d go for that before anything else.

I can try that, but it looks like they’re already there:

I think you’ll find that the specific one mentioned in that error message, and reproduced in my earlier post, is not in that list. There are some similar-looking ones, but likely not this one.

OK, got it that’s my fault. I’ll add it in - just so I’m clear, the addition goes in file extensions or media types or both?

Just in media types.

Thanks - got it trying the whole thing again now!

SUCCESS!
Thank you. That did the trick. Now to figure out how to import exhibits…