You're definitely misremembering. I may not be as old as some of you but I remember watching live fights on regular television. I even recall getting up early to watch live fights that were being fought in Japan (didn't Ali "fight" a Japanese wrestler at some point?).
There were live fights (Friday nights, etc.) but some 60s bouts were in movie theaters. They called it "closed-circuit".
EDIT: found this ...
A look back at the 1964 Clay-Liston closed-circuit telecast
Tuesday marks the 50th anniversary of the 2/25/1964 heavyweight boxing match in Miami between challenger Cassius Clay (now known as Muhammad Ali) and defending champion Sonny Liston.
Theater Network Television, Inc (TNT). produced the closed-circuit telecast of this fight and carried it live at 10 pm ET on a Tuesday night. The announcers on that telecast were Steve Ellis and former heavyweight champion Joe Louis. The production was shown in over 350 theaters and arenas throughout North America. TNT also made the telecast available via satellite to Europe on a tape delay of a few hours.
http://www.classictvsports.com/2014/02/a-look-back-at-1964-clay-liston-closed.html