Here's the thing. You have operators who derive their perceptions from actual combat experience and then you have bureaucrats who derive their perceptions from big think tanks like the RAND Corporation. Strategic decisions such as new weapons platforms are generally shaped by the bureaucrats and thus the big think tanks whose ideas may work in theory but fall apart in practice. This is what lead the military to adopt the F-4 as a major multipurpose workhorse.
Fighter planes from WWII forward were getting faster and faster to the point where it was believed that dogfighting would become obsolete because the pilot could not handle the burdens placed upon him by the extreme speeds. It was thought that missiles would do the dogfighting because they could theoretically react more quickly and handle a higher G-load. Thus the F-4 was designed mainly as a missile deployment platform. It was optimized to provide high speed in exchange for high rate of change (turn circles, Ps, acceleration, etc.) as well as other important attributes such as pilot visibility. While the F-4 was lauded by analysts as an evolution in aircraft design, it was generally denounced by fighter pilots as a turkey.
Most would agree that the fighter pilots were right. The missiles employed on the aircraft (early iterations of the AIM-4 Falcon, AIM-9 Sidewinder, and AIM-7 Sparrow) did not perform up to their expectations (malfunctioning most of the time). A clever enemy could exploit the weaknesses of the Phantom to force a dogfight and when it did happen the enemy fighter had an advantage over the F-4. This resulted in unprecedented losses (4:1 US kill ratio compared to 10:1 plus in previous conflicts). The losses could have been much higher had it not been for the generally large gap in pilot skills.
As far as UAV's go, it's generally believed that they can replace some of the more dangerous jobs traditionally flown by a human (such as flying the "hunter" in the hunter-killer setup on a SEAD strike). There are several vulnerabilities that might make this impractical against an enemy of equal or near-equal capability such as vulnerability to ECM, latency between pilot and machine, and decreased pilot awareness. When you start talking about AI-controlled platforms now you're bringing in a plethora of problems (namely, the inherent inability of current AI to perform on a level even close to that of a human). Only combat against a truly capable foe will prove whether UAV's will live up to their expectations and I think most will agree that UAV's will never entirely replace platforms with a pilot in the seat.
Here's one for you to research: the abandonment of the M1 Garand in favor of the M14 and then the M14 to the M16; two very controversial decisions at the time. Come to think of it, nearly all evolutions in standard-issue kit have had some degree of antagonism between operator and bureaucrat (another one is longbow->crossbow). Only history can tell who was right (and sometimes they're both right).
