Robert Milton Norby was born on May 5, 1919, in Havana, North Dakota. His parents' names were Alex and Selma Norby. He had five brothers and one sister; their names were Kenneth, Lester, Lloyd, Maloy, Gladys, and Wenzel. He spent all of his childhood in Havana. Robert attended Tewaukon Township School, a country school in North Dakota, but he could not finish high school because of work on the farm. After school he worked for Fred Bessler and Ted Hanson on neighboring farms. In his free time, he liked to help on the farm and play baseball with his brothers and sister.