What does this do and how do I use it? Nothing appears in the “page” dropdown in edit mode.
Which custom link do you try to use? They are links to special pages, from the core or from modules. The Search custom link is a link to the search engine that you have installed with the module Search.
Some other modules can add specific search pages too (a specific advanced form and results), like the module PslSearchForm, so you can select it when you build the navigation.
Note that the module Search alone doesn’t do anything, but is required by external search engines, like Solr. A basic mode for mysql/mariadb may be built in the future to allow this module to be usable.
Docs: https://omeka.org/s/docs/user-manual/sites/site_navigation/