Harvey was born in Sutton Coldfield, and studied at St John's College, Cambridge, eventually obtaining a PhD. He also took private lessons with Erwin Stein and Hans Keller on the advice of Benjamin Britten. Early musical influences included Schoenberg, Berg, Messiaen and Britten. While undertaking postgraduate study at Glasgow University, Harvey was a cellist in the BBC Scottish Symphony Orchestra. Around this time, he became interested in the music of Karlheinz Stockhausen. In 1969 he took up a Harkness Fellowship at Princeton University, where he encountered Milton Babbitt, another strong influence on his music. In the 1980s Harvey produced much music at IRCAM after receiving an invitation from Pierre Boulez to work there.
At IRCAM, Harvey produced works such as Speakings, a composition for large orchestra and electronics. The concept of the piece was to "make an orchestra speak". IRCAM is known for speech analysis and in this piece, special technology was developed to allow the analysis of speech to be realized in an orchestral context, using complex algorithms which can process multiple combinations possible in an orchestra setting. The program Orchidée computed such analyses and provided possible orchestrations for the composer.
Harvey was a Visiting Professor of Music at University of Oxford, Imperial College London, and an Honorary Professor at Sussex University.
From 2005 to 2008, Harvey held the post of Composer in Association with the BBC Scottish Symphony Orchestra.
In 2009 he was Composer in Residence at the Huddersfield Contemporary Music Festival. He died, aged 73, in Lewes.