Many companies create custom error pages to provide a more meaningful message to users if there is an issue with the website. IIS provides an easy way to add custom error pages in IIS Manager. However, using IIS Manager for adding a custom page for 401 errors may break Windows Authentication.
If your website keeps prompting credentials even though you enter the correct username and password, your custom error page might be the one causing the issue.
What to Do If The Custom Error Page for 401 Status Breaks Windows Authentication
This behavior is by design. It is expected for Windows Authentication to break if there is a new custom error page for 401 errors. 401 challenge is not designed to work with customized error pages.
You may try some workaround suggested on internet like the ones here but it’s unlikely that a new page will work with 401 challenge.
Instead of trying to add a new custom error page, I would recommend editing the IIS default error page located at %SystemDrive%\inetpub\custerr\en-US\401.htm
Simply open this file in notepad and make changes. You may want to enter a user-friendly message with contact information of the IT Support. Then save it as htm file. Replace the existing one.
If the sub-status code for 401 error is 2, the issue might be different. Check this post out for the solution: IIS binding limit (401.2 Windows Authentication error)