I am going to say that probably most people who play games never bother to vote on them on Steam or metacritic. This game is 100% a RPG, and it doesn't have to have deep dialogue or a good user interface to be an RPG - these are not prerequisites for being an RPG. These may be features that some people care about and personally prefer, but they are just not necessary for the game to be considered an RPG objectively. It has widely variable character builds, open-ended design, world reactivity, multiple solutions, and multiple endings depending on how you play the game.
To me personally, I feel like the combat is a lot of fun - the exploration is deep and rewarding, the side quests are often more than fetch and run and kill variety and tend to have more interesting stories to them than most other RPGs. The overall storyline is solid, and the character building is good too. I don't think that the loss of the stats reduced the overall depth of the game, just how you have to approach picking your perks. It still gives a lot of variety to how you want to build your character. If you don't like to build stuff, then you largely don't have to do it, but the base building and weapon customization is a lot of fun for me. If having these extra features makes Fallout 4 less of an RPG, then I guess Witcher 3 is just a card game because it has gwent in it.