Relic Retriever's last ability will check as your end step starts to see if any cards have left your graveyard this turn. If none did, the ability won't trigger at all. It's not possible to remove cards from your graveyard during your end step in time to have the ability trigger.