The Anatolian noun genders, some of its declension and most of the verb conjugation are very distinctive and unlike that of all non-Anatolian IE branches. Those are not just a few words that look "out of place", but the core structure that determines how the languages works and builds up into coherent sentences. I think most linguists consider Anatolian probably "archaic", in the sense of having undergone a very ancient divergence, because of syntactic matters, not because of its lexicon. The non-IE influence would have had to be extremely, actually unprecedentedly strong to change a lot of relevant parts of the very core of the language (the way its nouns and verbs work), and not just its vocabulary or phonetics.
I find that unlikely. I mean, the least complicated explanation is that, instead of Anatolian being an extremely non-IE-shifted Late PIE dialect, it simply branched off first when PIE was still very different and then had its own isolated evolution. No other IE language changed so profoundly just because of heavy foreign influence, not even those that most clearly seem to have suffered a lot of non-IE influence, like Greek, Germanic and Indo-Aryan.