I think they are both great albums, but personally, I listen to Relapse much more. I also think it is an overall better album, for several reasons.
1. Production
Dre, probably the best producer in the game, and Eminem have a deep-rooted chemistry. I know Eminem wanted to branch out and try new instrumentals, but producers like Just Blaze, Boi-1-da, etc. gave him a bunch of cliche soft-rock beats with annoyingly synthetic drum loops.
2. Storytelling
Eminem is a gifted storyteller in his rhymes. In relapse, he painted a portrait of a horror story and when he wasn't doing that, he painted a portrait of his life in recluse. In Recovery, yes, there are 3-4 songs that do this, but the rest are just free-associative freestyles. They are random. And he relies way too much on puns and metaphors, that weren't really done much in hiphop until Lil Wayne came in. I like the concept of "Not Afraid" for example, but the first verse wasn't really that profound, it was just a random silly freestyle. "Going Through Changes" was an amazing song, I wish there were more songs like that on there.
3. Uniformity
Eminem sounds the same in each song in Recovery. The accents in Relapse were silly, but at least it let him change up his flow anytime he wanted to. It just feels like most tracks in Recovery were the same: Eminem shouting in monotone over a cliche soft-rock drumloop beat.