Import media in CSV with File Sideload as private

ploc · January 30, 2020, 12:41am

I’m currently importing data from a CSV file with CSV Import plugin and there are image attached that are imported using File Sideload module.

However, I can figure out how to importe sideloaded files as private. My attached images are always public.

Is there something I have not see?

When importing, I’m ticking the Import values as private checkbox. But it looks like imported images are not strictly values and hence are not set as private.

jflatnes · January 30, 2020, 7:29pm

Yes, the “import values as private” box doesn’t affect anything but values.

I don’t think we currently have an option in the importer for making imported media private. You could of course make the item containing the file private, which will have the effect of making all the media for that item private as well.

Also, if you do an import of just Media (or a “mixed” import), so the whole line of the CSV is just for each media, you can set the public/private flag that way.

ploc · January 30, 2020, 8:05pm

I’m importing items (paintings) and 2 associated media:

the first one is a low-quality media and should be public
the second one is a high-quality media and should remain private

I can’t make the whole item private, as it is a public catalog, so I think I will be obliged to simply remove the high-quality media.

Does anyone see another better option?

Daniel_KM · January 31, 2020, 11:29am

You can set the default setting to private in the main settings, then batch process media after import, selecting only the ones you want to make public.

ploc · February 5, 2020, 9:17pm

I have around 3000 medias.

Can I select media based on their path?

I tried to select all media where source contains somepath, but no media is selected. Half of the media are saved under the path picture-original while the other half of the media are saved under the path picture-resized.

ploc · February 7, 2020, 9:29pm

I’m still looking for a simple way to import 2 media per item, where one item should remain public while the other media should be set as private.

jflatnes · February 10, 2020, 6:11pm

If you split up the media from the items, you could do this as two imports:

Make sure each item has some unique identifier that will be imported, then split out the media-related column(s) into another sheet, 1 row per media. That media sheet should include a reference to that unique identifier (so each media row is “pointing” at the item which is its parent). Add a column that specifies public vs private for each media (it can just say “public” and “private” in each cell of that column).

Import just the item data as an item import as normal. Then, do a “Media” import of the media-only sheet. You must map that item identifier column as “Item (by selected property)” under the Media-specific data section, and then you can map the visibility and media source columns, with the result that you can set the visibility of each imported media.

It’s also possible to interleave these into one sheet and do a “Mixed resources” import, but it’s a little fiddlier to set up and might not be worth the extra trouble.

ploc · February 16, 2020, 4:02pm

Thanks @jflatnes for your suggestion. I tried to apply it but I did not succeed. I think I made a mistake somwhere. Let me explain precisely what I’ve done, it might help us to find out what went wrong.

First, I imported my items and the media that should be public in the end. It worked well!

Then I tried to import another csv file that only contains the item id and the path to the media that should remain private. The job completed without error. Below is part of the logfile:

2020-02-16T12:02:29+00:00 INFO (6): 1 items were updated (append): "1" (6907).
2020-02-16T12:02:29+00:00 INFO (6): 1 items were updated (append): "2" (6908).
2020-02-16T12:02:29+00:00 INFO (6): 1 items were updated (append): "3" (6909).
2020-02-16T12:02:29+00:00 INFO (6): 1 items were updated (append): "4" (6910).
2020-02-16T12:02:29+00:00 INFO (6): 1 items were updated (append): "5" (6911).
2020-02-16T12:02:29+00:00 INFO (6): 1 items were updated (append): "6" (6912).
2020-02-16T12:02:30+00:00 INFO (6): 1 items were updated (append): "7" (6913).
2020-02-16T12:02:30+00:00 INFO (6): 1 items were updated (append): "8" (6914).
2020-02-16T12:02:30+00:00 INFO (6): 1 items were updated (append): "9" (6915).
2020-02-16T12:02:30+00:00 INFO (6): 1 items were updated (append): "10" (6916).
2020-02-16T12:02:30+00:00 INFO (6): 1 items were updated (append): "11" (6917).
[...]

The CSV Import past imports tab tells me that it worked well and that 100 items were updated:

Result:

Skipped: 1 items

Updated: 100 items
Status:

completed

So, it appears that everything seems to be ok.

But, in reality, I cannot see any changes on the items side. Each item has still only one media (the public one, imported during the first csv import), and the second media is missing (the one that should have been imported during the second csv import).

I imagine that medias have been imported somewhere but have not been attached to existing items.

Here how I configured the csv import:

click on Modules > CSV Import > Import
click on browse to select the csv file to import
click on Next
on the Map to Omeka S data tab,
- first colum is mapped to Item set (by selected property) > Dublin Core : Identifier
- second column is mapped to Media source > Sideload
- third column is mapped to Generic data > Visibility public/private
on the Basic Settings tab, I do not set anything (as I will set to append to existing item in next tab)
on the Advanced Settings tab,
- set Action to Append data to the resource
- set Resource identifier column to column dcterms:identifier
click on Import

Here is an extract of the csv file:

dcterms:identifier,Filename-originals,public/private
1,originals/1.jpg,private
2,originals/2.jpg,private
3,originals/3.jpg,private
4,originals/4.jpg,private
5,originals/5.jpg,private
6,originals/6.jpg,private
7,originals/7.jpg,private
8,originals/8.jpg,private
9,originals/9.jpg,private
10,originals/10.jpg,private

Any idea on what I should change?

jflatnes · February 18, 2020, 4:01am

Your second import, the media-only one: it looks like you’re leaving “Items” selected on the initial import screen, but I was suggesting you try choosing “Media” there. You should then have the option to map to “Item (by selected property)” (note, not “Item set” as you described). In this setup you shouldn’t need to touch the Advanced Settings tab.

ploc · February 18, 2020, 10:21pm

Thanks for your prompt response!

I tried to import my data following your new advices.

Here is the json configuration of the import job:

Status
In Progress
Started
February 18, 2020 at 11:13:26 PM GMT+1
Ended
[not ended]
Class
CSVImport\Job\Import
Owner
me@example.org
Args

{
    "filename": "import.csv",
    "filesize": "2837",
    "filepath": "\/srv\/data\/tmp\/omekaCHlmV9",
    "media_type": "text\/csv",
    "resource_type": "media",
    "comment": "",
    "automap_check_names_alone": true,
    "column-property": [
        {
            "dcterms:identifier": 10
        }
    ],
    "column-item": [
        "dcterms:identifier"
    ],
    "column-data-type": [
        "literal",
        "literal",
        "literal"
    ],
    "column-media_source": {
        "1": "sideload"
    },
    "column-is_public": {
        "2": "1"
    },
    "generic-data": "default",
    "media-source": "default",
    "o:resource_template": "",
    "o:resource_class": "",
    "o:owner": {
        "o:id": 1
    },
    "o:is_public": 1,
    "multivalue_separator": "|",
    "global_language": "",
    "action": "create",
    "identifier_column": 0,
    "identifier_property": "dcterms:identifier",
    "action_unidentified": "skip",
    "rows_by_batch": 20,
    "column-multivalue": [],
    "delimiter": ",",
    "enclosure": "\"",
    "escape": "\\"
}

Log
[no log]

Once completed, there is no log nor errors. But no media file seems to have been imported.

But the job was quite long, and the Past imports page says that Added: 100 media, so I think something has been added. But I can’t see where.

There is still something that I don’t understand.

Here is how I proceed:

click on Modules > CSV Import > Import
click on browse to select the csv file to import
set Import type to Media
click on Next
on the Map to Omeka S data tab, set all data as displayed in the screenshot below:
- first colum is mapped to Media-specific data > Item (by selected property) > Dublin Core : Identifier
- second column is mapped to Media source > Sideload
- third column is mapped to Generic data > Visibility public/private
on the Basic Settings tab, I do not set anything
on the Advanced Settings tab, I do not set anything either
click on Import

Here is the screenshot of the import configuration step:

Any idea on what I should change?

jflatnes · February 19, 2020, 2:09am

On the left-side admin sidebar, after clicking Items, there’s a subentry for Media that lists all the media in the Omeka S installation. Does that seem to show the “missing” media?

ploc · February 19, 2020, 7:48am

Oh yes, there are here…

I got 700 of them where I should only have 200 of them (2 x 100). This is the result of multiple successful imports. And I don’t even realized that it was working.

The problem that imported medias are not linked to my items. Is that normal? If yes, how could I create such links? (for memories, in the end, I’ll have to manage thousands of items, not one hundred, so I cannot afford manual linking).

jflatnes · February 19, 2020, 7:31pm

Media will always be linked to some item.

If you click the “details” icon (the ellipsis) on any of those media you’re seeing in that list, the sidebar will show “Part of Item” and present a link to the item the media is assigned to.

ploc · February 19, 2020, 8:27pm

Thanks a thousand of time @jflatnes for your kind help.

I managed to import my public and my private images.