Had the Gunners clinched all three points, they would have moved to fourth spot nearing rivals Chelsea and Manchester giants while the Canaries won their first match this season, moving out of the relegation zone.
Grant Holt put an to Norwich's 13-match winless run agaisnt the Gunners. Arsenal goalkeeper Vito Mannone couldn't hold on to Alex Tettey's thunderous shot and Holt tapped in from close range.
Arsene Wenger's side had 70 minutes to back into the game but Norwich were determined and defended to the throat to deny any chances for the Londoners.
Wenger denied it was Mannone's mistake which cost them the fourth spot but blamed the enitre team for the 'shock' loss but hopes it will have a 'positive impact' on his team.
"It was a shock for the team and I hope it will have a positive impact," Wenger said.
"Maybe we underestimated the difficulty we faced today. If you are not ready for a fight in the Premier League you will have all these bad surprises," he said.
Most players in the Gunners line-up were away on international duty during the week and may have lacked 100 percent fitness levels but Wenger said: "We had players away on international duty and some, including Santi Cazorla, were not at their best physically. But it's not an excuse. When you are better players you have to be able to cope."
Gunners lacked fluency in the final third despite all the possession but Norwich did not look as bad as in their previous seven games in which they conceded 17 goals.
Arsenal slipped to their second defeat this season and are currently placed ninth in the league with 12 points after eight games.