ssh-keygen should print an error message if it was unable to do its job,
and return non-zero.

$ wc .ssh/known_hosts
  16   48 6900 .ssh/known_hosts
$ ssh-keygen -R jidanni.org; echo $?
/home/jidanni/.ssh/known_hosts updated.
Original contents retained as /home/jidanni/.ssh/known_hosts.old
0
$ wc .ssh/known_hosts
  16   48 6900 .ssh/known_hosts

Just like rm does,
$ rm zzz; echo $?
rm: cannot remove `zzz': No such file or directory
1

Or the ssh-keygen man page should document at -R, that that is
intentional.

Also there should be a way to not just remove it, but instead update it,
all in one step.