Handling of records removed at source
We need to be able to handle records that are removed / not available anymore at the source.
I believe in previous discussions we contemplated something like "DELETED_AT_SOURCE" status, but we did not yet clarify in detail neither the mechanism to identify these records, nor the implications for version-handling.
It cannot be handled during the ingest, because the ingest only knows the records available at the source. Thus it has to be part of the continuous automatic checks, a routine trying to reach the item's source, and set the status accordingly if it is not available anymore at the source.