So, it seems the Saami were probably there first, speaking an unknown language, adopted a Finno-Ugric language from another group, and to some extent mixed with people coming north from the direction of Estonia also speaking a Finno Ugric language? Those people and I1 people from Scandinavia pushed the poor Saami far north?
Interestingly, years ago when I looked up what we know of the history of the Saami, before genetic analysis, I read that the Saami were a European population who had spoken a different language previously. Some words and placenames originated in the previous language. Most probably a language isolate. A migration from the east had imposed a different language on the Saami. Then scientists had come along and named the language and the language group for the people where they first found it. So the Saami language and language group is not actually indigenous to the Saami people. Or so the story went back then, before all this new information. The migration that replaced the language was said to have been about 500 AD, which is much later than this new estimate. Although fairly recent research indicates humanity went extinct, or very near extinct in northern Scandinavia in the 540s so it would be good timing. There would have been more than one migration I expect.
Anyway, the Saami being pushed further north is fairly well attested local history. Its not been a case of Saami territories and Non-Saami ones, though. More like timeshare. We're at the edge of agricultural viability and in a warmer climate the agriculturalists expanded. Cooler periods meant the reindeer herders strategy paid off better and they expanded, while agricultural terriotries contracted. Thats not been the pattern for the last few centuries though because the tech edge of the agriculturalists have grown larger.