Ah, I see, sorry for misunderstanding. GitHub returns 404 when access is denied. Please ensure that account from which you use token, has access to the repository with PR.
Another possible issue is double encryption of that token. For example you encrypted token with https://ci.appveyor.com/tools/encrypt, then pasted encrypted data to notifications settings UI and then exported to YAML. In this case token will be be double encrypted.