Import API stalling

cmccullough · June 25, 2018, 5:52pm

Hi,

I’m trying to import all items from an omeka.net site to a new server .org site, and the import stalls partway through. I’ve tried undoing the import and restarting, and for a while, the number of successful items was creeping higher (80, 106, 159 – out of about 300), but now it’s back to 106. Here is the error log for the last two attempts (159 and 106):

2018-06-25T15:06:04+00:00 WARN (4): Error output from ImageMagick:
GPL Ghostscript 9.07: Unrecoverable error, exit code 1
convert: Postscript delegate failed `/tmp/91167f8b2e6f1d3abb3b51d370a82d40.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/678.
convert: no images defined `/tmp/fullsize_91167f8b2e6f1d3abb3b51d370a82d40.jpg' @ error/convert.c/ConvertImageCommand/3046.

2018-06-25T15:06:04+00:00 ERR (3): ImageMagick failed with status code 1.
2018-06-25T15:06:51+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:06:52+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:06:53+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:07:43+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:07:43+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:07:44+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:14:19+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:14:19+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:14:20+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:14:44+00:00 WARN (4): Error output from ImageMagick:
   **** Warning: considering '0000000000 XXXXX n' as a free entry.

   **** This file had errors that were repaired or ignored.
   **** The file was produced by: 
   **** >>>> iPhone OS 11.2.5 Quartz PDFContext <<<<
   **** Please notify the author of the software that produced this
   **** file that it does not conform to Adobe's published PDF
   **** specification.


2018-06-25T15:14:45+00:00 WARN (4): Error output from ImageMagick:
   **** Warning: considering '0000000000 XXXXX n' as a free entry.

   **** This file had errors that were repaired or ignored.
   **** The file was produced by: 
   **** >>>> iPhone OS 11.2.5 Quartz PDFContext <<<<
   **** Please notify the author of the software that produced this
   **** file that it does not conform to Adobe's published PDF
   **** specification.


2018-06-25T15:14:46+00:00 WARN (4): Error output from ImageMagick:
   **** Warning: considering '0000000000 XXXXX n' as a free entry.

   **** This file had errors that were repaired or ignored.
   **** The file was produced by: 
   **** >>>> iPhone OS 11.2.5 Quartz PDFContext <<<<
   **** Please notify the author of the software that produced this
   **** file that it does not conform to Adobe's published PDF
   **** specification.


2018-06-25T15:33:13+00:00 WARN (4): Error output from ImageMagick:
GPL Ghostscript 9.07: Unrecoverable error, exit code 1
convert: Postscript delegate failed `/tmp/acfdd38424876d087d4a695142805496.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/678.
convert: no images defined `/tmp/fullsize_acfdd38424876d087d4a695142805496.jpg' @ error/convert.c/ConvertImageCommand/3046.

2018-06-25T15:33:13+00:00 ERR (3): ImageMagick failed with status code 1.
2018-06-25T15:34:55+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:34:56+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:34:57+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:36:19+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:36:19+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-25T15:36:20+00:00 WARN (4): Error output from ImageMagick:
   **** ERROR: Unable to process JPXDecode data. Page will be missing data.

I’ve read some of the other threads with similar issues, but as I’m relatively new to Omeka (particularly on the .org backend), I’ll need some pretty clear specifics of what to try and where I can make those changes. Thank you so much!

patrickmj · June 25, 2018, 8:10pm

It looks like Omeka is having a variety of troubles creating the derivative images. I’d start by trying to create an item the normal way, using an original file from one of the items that is not getting imported. That’s to check if it’s something about the file itself or some other thing on your server.

cmccullough · June 25, 2018, 8:33pm

Thank you for your response! I tried adding a test item with a local file, and that worked fine. However, Omeka’s display of the items (and this is a fairly new issue of the last few hours) is weird. It’s only showing one item per page, but says there are 117. You can access the others through workarounds like the collections, but there seems to be no way to see them on the items page. If you have any ideas on either of these issues, I’d really appreciate them.

patrickmj · June 25, 2018, 8:53pm

Well, that’s a new twist. I wouldn’t guess that they’re related, but stranger things have happened. Anything in the logs from when you tried the browse page?

cmccullough · June 25, 2018, 8:56pm

No, there’s nothing new in the log after refreshing the page, trying to sort, and trying to select all items.

jflatnes · June 25, 2018, 9:04pm

That page you’re showing is encountering an error, probably when trying to display the Creator field, as you can see the other table cells aren’t there and there’s no footer.

If you enable error display you should get a message there indicating what’s happening.

cmccullough · June 25, 2018, 9:16pm

Thank you so much! It found an error in one of my plugins, so I deactivated it and now it’s displaying the items fine. Though, that still leaves the issue with the import API, as it seems they aren’t related after all.

cmccullough · June 26, 2018, 6:41pm

Hi again,

I tried the import three times again today. The first two failed to start with the following message:

exception ‘Zend_Uri_Exception’ with message ‘Illegal scheme supplied, only alphanumeric characters are permitted’ in /home/rocheste/public_html/application/libraries/Zend/Uri.php:111
Stack trace:
#0 /home/rocheste/public_html/application/libraries/Zend/Http/Client.php(312): Zend_Uri::factory(’ http://communi…’)
#1 /home/rocheste/public_html/plugins/OmekaApiImport/controllers/IndexController.php(19): Zend_Http_Client->setUri(’ http://communi…’)
#2 /home/rocheste/public_html/application/libraries/Zend/Controller/Action.php(516): OmekaApiImport_IndexController->indexAction()
#3 /home/rocheste/public_html/application/libraries/Zend/Controller/Dispatcher/Standard.php(308): Zend_Controller_Action->dispatch(‘indexAction’)
#4 /home/rocheste/public_html/application/libraries/Zend/Controller/Front.php(954): Zend_Controller_Dispatcher_Standard->dispatch(Object(Zend_Controller_Request_Http), Object(Zend_Controller_Response_Http))
#5 /home/rocheste/public_html/application/libraries/Zend/Application/Bootstrap/Bootstrap.php(105): Zend_Controller_Front->dispatch()
#6 /home/rocheste/public_html/application/libraries/Zend/Application.php(384): Zend_Application_Bootstrap_Bootstrap->run()
#7 /home/rocheste/public_html/application/libraries/Omeka/Application.php(73): Zend_Application->run()
#8 /home/rocheste/public_html/admin/index.php(28): Omeka_Application->run()
#9 {main}

The third processed 120 items and gave me this:

2018-06-26T18:15:33+00:00 WARN (4): Omeka_Storage_Adapter_Filesystem: Tried to delete missing file ‘original/142b1b96f38c994da2251a74ac08032f.pdf’.
2018-06-26T18:20:44+00:00 WARN (4): Error output from ImageMagick:
GPL Ghostscript 9.07: Unrecoverable error, exit code 1
convert: Postscript delegate failed /tmp/5332203c9b2ae72611f1b700d6b89772.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/678. convert: no images defined/tmp/fullsize_5332203c9b2ae72611f1b700d6b89772.jpg’ @ error/convert.c/ConvertImageCommand/3046.

2018-06-26T18:20:44+00:00 ERR (3): ImageMagick failed with status code 1.
2018-06-26T18:22:11+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T18:22:12+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T18:22:13+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T18:23:44+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T18:23:45+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

Any ideas?

jflatnes · June 26, 2018, 8:59pm

The “invalid scheme” ones seem like they’re complaining about a space at the front of the URL you’re using. You can see it in the error output, there’s a space between the quotation mark and the “http”.

For the second, are there no other errors or messages? The error in the middle about the PDF doesn’t seem like it stopped the process since there were more messages afterwards.

cmccullough · June 26, 2018, 9:03pm

I undid and re-ran the import several times today since each time was gathering an increasing number of items, and it made it almost 2/3 through before quitting. Here is the whole error log after those invalid scheme ones:

2018-06-26T18:15:33+00:00 WARN (4): Omeka_Storage_Adapter_Filesystem: Tried to delete missing file ‘original/142b1b96f38c994da2251a74ac08032f.pdf’.
2018-06-26T18:20:44+00:00 WARN (4): Error output from ImageMagick:
GPL Ghostscript 9.07: Unrecoverable error, exit code 1
convert: Postscript delegate failed /tmp/5332203c9b2ae72611f1b700d6b89772.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/678. convert: no images defined/tmp/fullsize_5332203c9b2ae72611f1b700d6b89772.jpg’ @ error/convert.c/ConvertImageCommand/3046.

2018-06-26T18:20:44+00:00 ERR (3): ImageMagick failed with status code 1.
2018-06-26T18:22:11+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T18:22:12+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T18:22:13+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T18:23:44+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T18:23:45+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:12:04+00:00 WARN (4): Omeka_Storage_Adapter_Filesystem: Tried to delete missing file ‘original/df6c68140f7a8daab0828ebeef7d478f.JPG’.
2018-06-26T20:14:59+00:00 WARN (4): Error output from ImageMagick:
GPL Ghostscript 9.07: Unrecoverable error, exit code 1
convert: Postscript delegate failed /tmp/1f0c53778028f9750a623fc97858b2e9.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/678. convert: no images defined/tmp/fullsize_1f0c53778028f9750a623fc97858b2e9.jpg’ @ error/convert.c/ConvertImageCommand/3046.

2018-06-26T20:14:59+00:00 ERR (3): ImageMagick failed with status code 1.
2018-06-26T20:16:03+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:16:04+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:16:05+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:16:57+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:16:58+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:24:02+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:24:03+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:24:23+00:00 WARN (4): Error output from ImageMagick:
**** Warning: considering ‘0000000000 XXXXX n’ as a free entry.

**** This file had errors that were repaired or ignored.
**** The file was produced by:
**** >>>> iPhone OS 11.2.5 Quartz PDFContext <<<<
**** Please notify the author of the software that produced this
**** file that it does not conform to Adobe’s published PDF
**** specification.

2018-06-26T20:24:24+00:00 WARN (4): Error output from ImageMagick:
**** Warning: considering ‘0000000000 XXXXX n’ as a free entry.

**** This file had errors that were repaired or ignored.
**** The file was produced by:
**** >>>> iPhone OS 11.2.5 Quartz PDFContext <<<<
**** Please notify the author of the software that produced this
**** file that it does not conform to Adobe’s published PDF
**** specification.

2018-06-26T20:24:25+00:00 WARN (4): Error output from ImageMagick:
**** Warning: considering ‘0000000000 XXXXX n’ as a free entry.

**** This file had errors that were repaired or ignored.
**** The file was produced by:
**** >>>> iPhone OS 11.2.5 Quartz PDFContext <<<<
**** Please notify the author of the software that produced this
**** file that it does not conform to Adobe’s published PDF
**** specification.

2018-06-26T20:32:00+00:00 WARN (4): Omeka_Storage_Adapter_Filesystem: Tried to delete missing file ‘original/2dfd305715af201b54dd707400355ae5.JPG’.
2018-06-26T20:34:00+00:00 WARN (4): Error output from ImageMagick:
GPL Ghostscript 9.07: Unrecoverable error, exit code 1
convert: Postscript delegate failed /tmp/987b8572e8cf04961d42fd31378a4141.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/678. convert: no images defined/tmp/fullsize_987b8572e8cf04961d42fd31378a4141.jpg’ @ error/convert.c/ConvertImageCommand/3046.

2018-06-26T20:34:00+00:00 ERR (3): ImageMagick failed with status code 1.
2018-06-26T20:34:47+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:34:48+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:34:50+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:35:09+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:35:10+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:35:44+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:35:45+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:35:46+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:35:47+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:43:46+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:43:47+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:43:48+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:43:49+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:44:07+00:00 WARN (4): Error output from ImageMagick:
**** Warning: considering ‘0000000000 XXXXX n’ as a free entry.

**** This file had errors that were repaired or ignored.
**** The file was produced by:
**** >>>> iPhone OS 11.2.5 Quartz PDFContext <<<<
**** Please notify the author of the software that produced this
**** file that it does not conform to Adobe’s published PDF
**** specification.

2018-06-26T20:44:08+00:00 WARN (4): Error output from ImageMagick:
**** Warning: considering ‘0000000000 XXXXX n’ as a free entry.

**** This file had errors that were repaired or ignored.
**** The file was produced by:
**** >>>> iPhone OS 11.2.5 Quartz PDFContext <<<<
**** Please notify the author of the software that produced this
**** file that it does not conform to Adobe’s published PDF
**** specification.

2018-06-26T20:44:10+00:00 WARN (4): Error output from ImageMagick:
**** Warning: considering ‘0000000000 XXXXX n’ as a free entry.

**** This file had errors that were repaired or ignored.
**** The file was produced by:
**** >>>> iPhone OS 11.2.5 Quartz PDFContext <<<<
**** Please notify the author of the software that produced this
**** file that it does not conform to Adobe’s published PDF
**** specification.

2018-06-26T20:55:46+00:00 WARN (4): Omeka_Storage_Adapter_Filesystem: Tried to delete missing file ‘original/9c7a0dba6ae2863cbafe6b329efd0cb1.JPG’.
2018-06-26T20:55:46+00:00 WARN (4): Omeka_Storage_Adapter_Filesystem: Tried to delete missing file ‘original/a8bb74877babf487d824c23a3fc9181b.JPG’.
2018-06-26T20:56:51+00:00 WARN (4): Error output from ImageMagick:
GPL Ghostscript 9.07: Unrecoverable error, exit code 1
convert: Postscript delegate failed /tmp/f947786edeb5870acb1d57f04bf9b3e8.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/678. convert: no images defined/tmp/fullsize_f947786edeb5870acb1d57f04bf9b3e8.jpg’ @ error/convert.c/ConvertImageCommand/3046.

2018-06-26T20:56:51+00:00 ERR (3): ImageMagick failed with status code 1.
2018-06-26T20:58:41+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:58:42+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T20:58:43+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T21:00:15+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

2018-06-26T21:00:16+00:00 WARN (4): Error output from ImageMagick:
**** ERROR: Unable to process JPXDecode data. Page will be missing data.

cmccullough · June 27, 2018, 3:02pm

Hi again,

I just wanted to let you know that I found a workaround in the comments here: https://community.reclaimhosting.com/t/migrating-from-omeka-net-to-omeka-on-reclaim-hosting/386

But thank you for your help!