Defense seem to be a lagging indicator for scouts (meaning decent expectations/performance for a player to play premium positions tend to stick far too long while poor expectations/performance also stick far too long). People are just starting to understand that Torri Hunter can't play CF anymore, 5 years after he started costing runs out there. Meanwhile, I think for Norris that first year where he was just starting to play the position and had tons of passed balls has really stuck in the minds of scouts.
But last year, Norris through out 51% of baserunners, and dropped his passed balls to 1 for every 11.5 games from one every 3.5 games. That's not quite Wilson Ramos defense, but it's not far off. Plus, Keith Law, who tends to be more conservative than most scouts in terms of projecting players to stay at premium defensive positions, was pretty definitive in projecting him to stay behind the plate.
Overall, I think it's far more likely that this will go the opposite way - that Norris' defense will actually turn into a positive - than that he will have to move from C. His improvement has been pretty rapid considering his inexperience at the position.