The image was created in 1952 by jazz musician Al Fairweather when he was a student at Edinburgh College of Art.
It shows the actor before he found fame as James Bond, posing in body-building trunks as a life model.
Curators chose the piece to be among work in the college's centenary exhibition - Ten Decades at the City Art Centre in Edinburgh.
The oil on canvas painting is just one of many Sir Sean, now 77, posed for over several years as a life model for the college.
The work was painted just before he auditioned for a stage production of South Pacific, a role which led to stage, TV and film work and the role as James Bond that propelled him to stardom.
The painting has been in the collection of the school of drawing and painting at Edinburgh College of Art for more than 50 years, but has rarely gone on display.
Co-curator Selina Skipwith said that while the picture of Sir Sean was popular, it was not a key piece in the exhibition.
She said: 'There has already been a lot of interest in Sir Sean Connery and it's quite funny for us because Ian Fleming, the creator of James Bond, was actually the grandson of our founder Robert Fleming, and next year there will be an exhibition of Ian Fleming's work.'
The exhibition also includes works from artists including Samuel Peploe and the Scottish colourist Anne Redpath.
Text and image by BBC news, UK