CSV Media import fails with sideload

Hi There,
I can import fine using CSV but have not been able to import any media other than manually via sideload. Every time I try via the CSV import the logs say: (internal_id) is not a valid item identifier. The same for any other field I choose instead of internal ID.

2022-07-05T17:54:40+00:00 ERR (3): “27272 Ad 1977 .jpg” (dcterms:identifier) is not a valid item identifier.
2022-07-05T17:54:40+00:00 ERR (3): “25540 683.jpg” (internal_id) is not a valid item identifier.
2022-07-05T17:54:38+00:00 ERR (3): “2615 068.jpg” (foaf:img) is not a valid item identifier.

What am I doing wrong, I’ve spent hours playing with this and am totally frustrated.!!!

Can you share what your mapping looks like? A screenshot would work.

From these error messages it looks like you’re mapping the column that contains filenames in some odd way, like as a lookup for existing items to map to a property or something like that.

The “Media source” mapping options are what you want to look for to add media through CSV Import. There should be an option for using File Sideload there, if that’s what you’re trying to do.

advanced_settings


page 1
basic settings

Okay, so there’s a couple things here. First is, the “media” import type is useful only for adding media to already-existing items. If that’s what you want to do, that’s good. If you want to instead create new items and media at the same time, you probably want to do an “Item” import. You can still pick “Media source” when importing items, and it will create the media and attach it to the newly-created item for the same row. Doing the import this way from the start is definitely a simpler way to get items and media created from a single CSV.

Now, about doing a “Media” import like you’ve shown here. That type of import just creates (or updates) Media objects, so it can’t create new items, it can only add to existing ones. It can make sense to use when you’ve got items and their metadata created already, and you just need to add the media.

First, to create new media you want to have the “Action” in Advanced Settings set to Create. The “Append” action for a media import would be used instead to modify existing media (to change Media metadata, for example).

Second, when you’re doing a Media import, there are two mappings you need to have, at a minimum. Those are “Item” (data that identifies the parent item the new media should be created under) and “Media source” (the URL, sideload filename, etc.). You have them both here in your screenshot, but where I think you’re going wrong is that you have mapped both of those to the same column. That’s pretty unlikely to be the right thing to do.

When you map “Item” and pick “Internal id” you’re saying that the data in the CSV column you’re mapping to contains the Omeka S ID number of the item you want to use as the parent. Instead it looks like “Photo 1” is probably a filename. It’s an appropriate column for mapping to “Media source” but probably not for “Item.”

What you need to pick here and which column you should use is dependent on your data, but “dcterms:identifier” is a common choice (so, if when you imported the items the first time you had a unique identifier in the CSV that you mapped to dcterms:identifier, then you could use that same column here and map it as “Item [dcterms:identifier]” to match up your imported media with the corresponding items you already imported before from the same rows.

ok, thanks for the prompt assistance.

I’ve tried again and can confirm that I am adding media to existing items in the system that do not have any media currently.

It’s getting further but logs now show this:

2022-07-06T02:27:32+00:00 ERR (3): exception ‘Omeka\Api\Exception\ValidationException’ in /home/acc/public_html/history/application/src/Api/Adapter/AbstractEntityAdapter.php:625
Errors:
{
“o:ingester”: [
“Media must set an ingester.”
]
}
Stack trace:
#0 /home/acc/public_html/history/application/src/Api/Adapter/AbstractEntityAdapter.php(341): Omeka\Api\Adapter\AbstractEntityAdapter->hydrateEntity(Object(Omeka\Api\Request), Object(Omeka\Entity\Media), Object(Omeka\Stdlib\ErrorStore))
#1 /home/acc/public_html/history/application/src/Api/Manager.php(224): Omeka\Api\Adapter\AbstractEntityAdapter->create(Object(Omeka\Api\Request))
#2 /home/acc/public_html/history/application/src/Api/Manager.php(78): Omeka\Api\Manager->execute(Object(Omeka\Api\Request))
#3 /home/acc/public_html/history/application/src/Api/Adapter/AbstractEntityAdapter.php(390): Omeka\Api\Manager->create(‘media’, Array, Array, Array)
#4 /home/acc/public_html/history/application/src/Api/Manager.php(227): Omeka\Api\Adapter\AbstractEntityAdapter->batchCreate(Object(Omeka\Api\Request))
#5 /home/acc/public_html/history/application/src/Api/Manager.php(97): Omeka\Api\Manager->execute(Object(Omeka\Api\Request))
#6 /home/acc/public_html/history/modules/CSVImport/src/Job/Import.php(355): Omeka\Api\Manager->batchCreate(‘media’, Array, Array, Array)
#7 /home/acc/public_html/history/modules/CSVImport/src/Job/Import.php(255): CSVImport\Job\Import->create(Array)
#8 /home/acc/public_html/history/modules/CSVImport/src/Job/Import.php(194): CSVImport\Job\Import->processBatchData(Array)
#9 /home/acc/public_html/history/application/src/Job/DispatchStrategy/Synchronous.php(34): CSVImport\Job\Import->perform()
#10 /home/acc/public_html/history/application/src/Job/Dispatcher.php(105): Omeka\Job\DispatchStrategy\Synchronous->send(Object(Omeka\Entity\Job))
#11 /home/acc/public_html/history/application/data/scripts/perform-job.php(66): Omeka\Job\Dispatcher->send(Object(Omeka\Entity\Job), Object(Omeka\Job\DispatchStrategy\Synchronous))
#12 {main}
2022-07-06T02:27:32+00:00 ERR (3): exception ‘Omeka\Api\Exception\ValidationException’ in /home/acc/public_html/history/application/src/Api/Adapter/AbstractEntityAdapter.php:625

OK that’s a problem for sure.

Can you try one thing: does it work differently/better if you map only one column as “Media source” instead of two?

same problem :frowning:

help

Can you share the versions of Omeka S, CSV Import, and File Sideload that you’re using?

Using current versions, I can’t reproduce your issue.

CSV import v2.3.1
Sideload v1.5.0
Omeka S v3.2.1

Happy to give you access to troubleshoot or do a video session?

Omeka S is installed in a directory called history
would that make any difference?

What are you using to map the record to the media file? are you configuring any of the other settings on the basic tab

Those versions all look fine.

The folder you’re installed under, that shouldn’t matter at all, no problem there.

I think what may be happening here is that you have some rows where the column you’re mapping as Media source is blank? That could cause this error. If you’ve got some rows with blanks like that, try doing an import with all those rows removed.

ok, I’ve removed the blank lines -

now getting this error which looks like it cannot find the file path as it’s all weird. Given sideload is working without the import I assume permissions are correct. It also is set to delete the file after loading which it does with a manual add file.

2022-07-08T23:34:56+00:00 ERR (3): exception ‘Omeka\Api\Exception\ValidationException’ in /home/acc/public_html/history/application/src/Api/Adapter/AbstractEntityAdapter.php:649
Errors:
{
“ingest_filename”: [
“Cannot sideload file “/home/acc/public_html/history/modules/FileSideload/Import_Files/1668 005.jpg”. File does not exist or does not have sufficient permissions”
],
“o:resource_template_property”: [
“The “Alvis Car History” resource template requires a “Import Master Key #” value”
]
}
Stack trace:
#0 /home/acc/public_html/history/application/src/Api/Adapter/AbstractEntityAdapter.php(341): Omeka\Api\Adapter\AbstractEntityAdapter->hydrateEntity(Object(Omeka\Api\Request), Object(Omeka\Entity\Media), Object(Omeka\Stdlib\ErrorStore))
#1 /home/acc/public_html/history/application/src/Api/Manager.php(224): Omeka\Api\Adapter\AbstractEntityAdapter->create(Object(Omeka\Api\Request))
#2 /home/acc/public_html/history/application/src/Api/Manager.php(78): Omeka\Api\Manager->execute(Object(Omeka\Api\Request))
#3 /home/acc/public_html/history/application/src/Api/Adapter/AbstractEntityAdapter.php(390): Omeka\Api\Manager->create(‘media’, Array, Array, Array)
#4 /home/acc/public_html/history/application/src/Api/Manager.php(227): Omeka\Api\Adapter\AbstractEntityAdapter->batchCreate(Object(Omeka\Api\Request))
#5 /home/acc/public_html/history/application/src/Api/Manager.php(97): Omeka\Api\Manager->execute(Object(Omeka\Api\Request))
#6 /home/acc/public_html/history/modules/CSVImport/src/Job/Import.php(355): Omeka\Api\Manager->batchCreate(‘media’, Array, Array, Array)
#7 /home/acc/public_html/history/modules/CSVImport/src/Job/Import.php(255): CSVImport\Job\Import->create(Array)
#8 /home/acc/public_html/history/modules/CSVImport/src/Job/Import.php(194): CSVImport\Job\Import->processBatchData(Array)
#9 /home/acc/public_html/history/application/src/Job/DispatchStrategy/Synchronous.php(34): CSVImport\Job\Import->perform()
#10 /home/acc/public_html/history/application/src/Job/Dispatcher.php(105): Omeka\Job\DispatchStrategy\Synchronous->send(Object(Omeka\Entity\Job))
#11 /home/acc/public_html/history/application/data/scripts/perform-job.php(66): Omeka\Job\Dispatcher->send(Object(Omeka\Entity\Job), Object(Omeka\Job\DispatchStrategy\Synchronous))
#12 {main}
2022-07-08T23:34:56+00:00 ERR (3): exception ‘Omeka\Api\Exception\ValidationException’ in /home/acc/public_html/history/application/src/Api/Adapter/AbstractEntityAdapter.php:649

ok I ditched the resource template and tried again but still no luck.

error attached

Yeah, the resource template was causing the other problem, and you probably didn’t want to have it set the same way for your Media as Items anyway.

These errors… so you’re sure the files are there under these exact names? With the space(s)? Shown in monospace font it looks to me like those names that are shown in the error message actually have two spaces in a row, which would mean there’s similarly two spaces in the sheet/CSV. If the actual files only have one space (or none, or any other number) you’d have this problem.