Jamie Cureton's second half volley wasn't enough to prevent Accrington Stanley taking all three points in Sky Bet League Two.
The Robins fate was sealed in the first half as the visitors went into the half time break with a two goal lead.
Bowerman opened the scoring with a cushioned header and turned provider for Stanley's second as Joyce's header doubled their lead.
Yates' men came out fighting in the second half and Cureton almost inspired a fightback but it wasn't to be and Cheltenham were unable to prevent their third straight defeat.
Terry Gornell returned to Mark Yates' starting eleven to face his former employer. The striker joined Stanley on loan in 2008 and returned on a permanent basis during 2010/2011.
Jamie Cureton was also named in the first team and the two forwards replaced Byron Harrison and Sam Deering as Yates reverted to a flat four in midfield.
James Beattie also made two changes. as expected Laurence Wilson started at left back following the departure of Lee Naylor earlier in the week and George Bowerman replaced Danny Webber.
Kayode Odejayi returned to the Abbey Business Stadium and started against the club he helped guide to promotion via the League Two Play-Offs in 2006.
There wasn't much to separate the two sides during the opening exchanges and both teams saw strikes from Bowerman and Richards curl high and wide of goal.
As the game developed, the Robins struggled to maintain a foothold into the tie and Wilson was able to pounce onto a defensive error and force Brown into a strong save.
Wilson tested the keeper again as he stung the palms of Brown when Yates' men failed to clear a poor free kick.
And the inevitable break through came after just 15 minutes as George Bowerman found space between Troy Brown and Craig Braham-Barrett to cushion his header low past Scott Brown, Nicky Hunt with the crossed assist.
Kal Naismith was given time on the edge of the box to fire a shot over the bar and forced Yates to revert to a diamond formation in an attempt to retain more possession.
Matt Richards' direct free kick looped disappointingly over the bar as the first half looked to be dragging to a close. Despite Stanley's one goal advantage and better attempts on goal, there wasn't much between the sides.
Jermaine McGlashan looked set to turn the tide. The winger rans 30 yards, cutting his path across the pitch and into the box, his shot looked destined for the roof of the net but for a fine Marcus Bettinelli save, his first of the afternoon.
And with only three minutes of the half remaining, Robins fans had their hearts in their mouth as Richards long ball found Cureton on the edge of the box. The striker took one touch to compose himself before unleashing a swerving strike that bounced back off the crossbar.
But the Robins were immediately brought back down to earth with a bump.
The visitor's countered and Bowerman's cross from the byline founds the head of Joyce running into the box to double the visitors lead.
The last action of the half that summed up the first 45 minutes.
Half Time: Cheltenham Town 0-2 Accrington Stanley
The half-time break worked wonders for the Robins, as the same eleven men started the second half much brighter than they ended in the first.
Roofe curled an effort wide of the far post after receiving a short corner and Gornell and Cureton both forced Bettinelli into action in the opening exchanges.
Cureton looked especially dangerous, finding space in and amongst the back four with excellent movement. He received the ball in space, got the better of two defenders but saw his strike sail over the roof of the LMI terrace.
With 30 minutes remaining, Yates introduced Deering and Harrison and they almost achieved instant results.
Roofe lunged well to get on the end of Deering's cross but leaning backwards the striker scooped the ball over the bar from 2-yards out. And moments later he was denied an almost a certain goal by a fine fingertip save.
The Robins resurgence was rewarded with 20 minutes remaining. Jamie Cureton's left footed volley into the roof of the net ensured an exciting climax.
Roofe was on hand again to sting the palms of Bettinelli with a strike from the edge of the box, Steve Elliott headed over from six yards and Roofe fired a left footed shot wide of goal as the Robins searched for a leveller.
However, the equaliser never emerged and Yates' men were condemned to their third straight defeat.
Full Time: Cheltenham Town 1-2 Accrington Stanley
Cheltenham Town FC: S Brown (GK); Jombati, T Brown, Elliott, Braham-Barrett; McGlashan (Vincent 74), Taylor (Deering 62), Richards, Roofe; Gornell (Harrison 62), Cureton
Subs not used: Roberts (GK), Ihiekwe, Hanks, Kotwica.
Accrington Stanley FC (4-4-2): Bettinelli; Hunt, Aldred, Winnard, Wilson; Naismith, Joyce, Murphy, Mingoia (Hatfield 80); Bowerman (Webber 74), Odejayi (Atkinson 76).
Subs not used: Liddle, McCartan, Gray, Windass.
Referee: Mr. Tim Robinson
Attendance: 2,648 (72 Visiting Supporters)