I probably missed someone, however, I did my best....
For a number of reasons I could any number of the actors:
Barbara Bel Geddes... To me she was the backbone, heart, soul and spirit of Dallas. I know Larry Hagman was considered the "star" of the show, however, BBG was the sky that star hung in.
Patrick Duffy... The Dream Season left JR listless. JR wasn't himself at all and when JR isn't JR Dallas wasn't much fun. I know there was a new crop of writers for that season and it showed. It was like they didn't really know the characters. Duffy was so vital to Dallas, the contrast between JR and Bobby was one of the best parts of the show. Plus, Duffy had chemistry with the lampshades and ottomans. He was missed terribly.
Victoria Principal... I know die hard Principal fans say that the reason the ratings started to take a hard dip was because of her departure. While I think she had a HUGE fan base, I don't think it was that much bigger than Duffy's, Hagman's or Gray's. Pam did leave a void, however, it wasn't as dire to the continuity of the show like Duffys or to a lesser extent Grays was.
Linda Gray... Without Sue Ellen, JR was incomplete. Just like JR was listless after Bobby left, he was the same way after Sue Ellen left. I have to say I missed Gray in the same way as I missed Duffy. I do agree with Linda Gray's quote about the decision to leave Dallas by saying "Sue Ellen had come full circle". I just wish we would've had a bigger circle, so to speak.
Charlene Tilton... I, honestly, don't know how in the world the writers who created such iconic characters as JR, Sue Ellen, Bobby, Jock, Ellie, Cliff and Pam couldn't figure out what on Earth to do with Lucy. Tilton may not have had the big stories, however, she was such a major force and had some of the best zingers. Only JR had better one liners in my opinion.
That's all the opining I am going to do. I am not voting quite yet in my own poll. I am torn between BBG, Duffy and Gray. Those three, IMO, left such voids. Luckily they all came back in their own time. I'll vote in a few days....