There are still games being released in the Total War series, the latest being Pharaoh. You might want to start with Total War: Rome 2, which goes on sale for 75% every so often.
As for games like Company of Heroes, the latest Company of Heroes game wasn't reviewed very well and I don't know of any other recent successful RTS games that are similar. The best alternative seems to be Men of War: Assault Squad 2, which is 10 years old now.
For Red Alert 3 type games, I think the most popular right now is Age of Empires 4, besides the remakes of Age of Empires 2 and 3 and the remake of Age of Mythology.
You could check out the list of RTS games sorted by current player numbers to see which games are (still) popular. Though note that I wouldn't describe all of the games on this list as RTS:
Edit: I should also mention Starcraft 2, which wouldn't show up on this list as it isn't on Steam. The multiplayer and Terran campaign are available for free, but you'll have to pay to get the Protoss and Zerg campaigns.
There was also a recent article on the main site about how there aren't any modern RTS games that seem to be able to live up to the RTS games of old:
The RTS renaissance never happened and probably never will.
www.pcgamer.com