We’ve been notified by Google that there are some issues with our Omeka installation: 52 files have been “indexed although blocked by the robots.txt file”.
I’ve checked, and found that all those files are included in the directory files/original of our repository; the robots.txt file, as per Omeka installation, is very simple and indeed blocking access to that directory:
User-agent: * Disallow: /files/
I suppose those 52 files are linked from outside the repository, and therefore have been indexed by Google.
Is there anything that could be done to solve the Google issues, without causing problems to the Omeka installation?