Luckman grew up in Brooklyn, the child of German-Jewish immigrants. He excelled in football at Erasmus Hall High School and as a senior faced active recruitment by dozens of college teams. He chose Columbia after meeting legendary Lion coach Lou Little at a Columbia-Navy game at Baker Field. Little used Luckman, who also played defence, as a passing tailback and as his punter. In 1938, his senior season, Luckman defeated Yale virtually single-handedly; he also led a thrilling comeback victory over Army, a game he would later recall as his favorite. In 24 collegiate contests, Luckman ran up impressive statistics: 180 pass completions in 376 attempts, for 2,413 yards passing and 20 touchdowns. Honored as a second-team All-American, Luckman finished third in balloting for the Heisman Trophy, despite Columbia having a losing season. After graduating from Columbia College, he was chosen second overall in the 1939 professional draft.

Luckman also played baseball at Columbia and participated in the first sporting event ever broadcast on television—a game against Princeton at Baker Field on May 17, 1939. He graduated with his class that spring. In the late 1940s he returned to Columbia to help Coach Little teach the T-formation strategy to the Lions.