ref
From cave + man, based on the supposition that early humans dwelt in caves.
caveman (n.)
also cave-man, "prehistoric human dwelling in a natural cave," 1865, from cave (n.) + man (n.). Related: Cave-woman (1904).