David Alexander Colville was born on August 24, 1920, in Toronto, Ontario, the second son of Scottish immigrant David Harrower Colville and his wife Florence Gault. His father worked in construction, and the family moved to St. Catharines, Ontario, in 1927, and then to Amherst, Nova Scotia, in 1929. Soon after the move to Nova Scotia, Colville contracted pneumonia, and during his six-month recovery his mother supplied him with art materials, fostering his blossoming interest in art. During this period, he exclusively drew machines, cars, and boats. In 1934, Colville began taking weekly art classes under Sarah Hart, who taught in a Post-Impressionist style as an extension of Mount Allison University's fine arts department. In 1938, Colville enrolled at Mount Allison University, where he studied with Canadian Post-Impressionists including Stanley Royle and Sarah Hart, graduating with a Bachelor of Fine Arts in 1942.

Colville married Rhoda Wright, who he had been friends with since his first year at Mount Allison, in 1942 and enlisted in the Canadian Infantry shortly afterward. He served as a lieutenant in Fredericton, New Brunswick, and Camp Petawawa, Ontario. After being in the army for two years, and because he was a fine-arts student, he was appointed an Official War Artist in May 1944. He painted in Yorkshire and participated in preparations for D-Day, taking part in the Royal Canadian Navy's landings in southern France. His unit, attached to the 3rd Canadian Division, relieved the 82nd Airborne Division at Nijmegen, Netherlands, in mid-September 1944 during Operation Market Garden and remained there until the following February. He continued on tours in the Netherlands and Germany, where he was also tasked with depicting the horrors of the Bergen-Belsen concentration camp, which he visited in October 1945 after its liberation. His painting "Bodies in a Grave" (1946), a scene of emaciated corpses in a Bergen-Belsen burial pit, is based on images he captured with his army-issue camera. He returned to Ottawa late in 1945 and worked on paintings based on his European sketches and watercolours until his demobilization in 1946.

After the war, Colville had already achieved some success, having exhibited at the Art Association of Montreal in 1941 and the Royal Canadian Academy of Arts in 1942. He became a faculty member in the Fine Arts Department at Mount Allison University, where he taught from 1946 to 1963. After settling in Sackville with his family, he concentrated on teaching and was influenced during this period by ancient Egyptian art that featured humans in frontal and profile positions. His 1950 painting "Nude and Dummy" marked the transition from the reportage of his war pictures to a personal creative direction. In 1951, Colville had his first solo show at the New Brunswick Museum in Saint John, which included landscapes of the Maritimes and was influential as the first time his work was written about in newspapers, the first time he lectured publicly on his art, and his first museum sale. Between 1952 and 1955, the Hewitt Gallery in New York gave Colville his earliest commercial exhibitions, and the National Gallery of Canada acquired seven of his paintings in the 1950s.

Colville is often cited as belonging to the Magic Realism movement, and within this he developed his own style that influenced both a regional and national art community as teacher and founder of what became known as Maritime Realism. By contrast to other members of the Maritime school, his compositions involved precise geometry. He influenced numerous students who pursued a realist painting style, including Tom Forrestall (class of 1958), Christopher Pratt (class of 1961), and Mary Pratt (class of 1961). In 1963, Colville resigned from Mount Allison to devote himself to painting and printmaking full-time from a studio in his home in Sackville on York Street. He changed his medium several times over his career, from oil to tempera to oil and synthetic resin, and after 1963 to acrylic polymer emulsion. He followed a long, careful process for each composition, taking precise measurements and proportioning these to an underlying geometric scheme, working on only one composition at a time and producing only three or four paintings or serigraphs a year since the 1950s.

Colville often used his immediate surroundings as subject matter, using his family as models. Throughout his career, his wife Rhoda Colville, who was also an artist and poet, served as a model for numerous celebrated works. Rhoda's recreational life as a swimmer, skater, dancer, singer, pianist, and cyclist was featured in his pieces, often set in the landscapes and waterways of Annapolis Valley. His paintings are characterized by a latent anxiety and are rigorously constructed according to precise geometry, executed with a technique consisting of minuscule dabs of paint applied meticulously dot by dot, with layers of thinned paint applied to primed wooden panels and finally sealed with transparent lacquer—a process that often took months. His subject matter was invariably chosen from his immediate environment but was never simply a recording of the everyday; rather, his highly representational reflections depicted a world filled with both the joyful and beautiful and the disturbing and dangerous.

In 1965, Colville designed circulation coins for Canada's centennial, allowing his work to reach the largest possible audience. In 1966, works by Colville along with those of Yves Gaucher and Sorel Etrog represented Canada at the Venice Biennale. He was made an Officer of the Order of Canada in 1967, elevated to Companion in 1982, the order's highest level. He served as visiting professor at the University of California at Santa Cruz in 1967-68 and spent six months as a visiting artist in Berlin in 1971. In 1973, he moved his family to his wife's hometown of Wolfville, Nova Scotia, where they lived and worked in the house that her father built and in which she was born. He was appointed chancellor of Acadia University in 1981, serving in that role until 1991.

Major retrospectives of Colville's work were held at the Art Gallery of Ontario in 1983, which subsequently toured Canada, Germany, and the Far East including Japan, making it the first time an exhibition of work by a living Canadian artist had been seen in that country. A major exhibition of his work done since 1984 was held at the Montreal Museum of Fine Arts in 1994, and in 2000 an exhibition was mounted at the National Gallery of Canada to mark his 80th birthday. In 2003, the Art Gallery of Nova Scotia organized the touring exhibition "Alex Colville: Return, Paintings Drawings and Prints 1994-2002." He designed the Governor General's Medal in 1978 and received the Canada Council Molson Prize in 1974. In 2003, he received the Governor General's Award in Visual and Media Arts. His work is held in major collections including the National Gallery of Canada, the Art Gallery of Nova Scotia, the Museum of Modern Art in New York, and the Centre Pompidou in Paris.

Colville and his wife Rhoda, who died on December 29, 2012, after succumbing to dementia, were married for 70 years and had one daughter and three sons, along with eight grandchildren and two great-grandchildren. Alex Colville died on July 16, 2013, at his house in Wolfville at age 92, after suffering from a heart condition. At the time of his death, the Art Gallery of Ontario was mounting "Alex Colville," the largest exhibition of the artist's work to date, which opened in 2014.