Well, you bring slaves home and you bring their genes as well. Look at North Africa.
I have a YFull SNP Match with an Ola Johnson Viktil, b. 1749 Trondelag, Norway, likely through a common ancestor who lived in Northern Ireland (The Ulaid) c. 650 CE (give or take a couple hundred years). He shares a common ancestor with two Byrnes c. 750 CE. His "Irish" ancestor could have been a captive who was taken to the Trondelag, and then sired Norwegian progeny.
*I2a-Y4751 ~450 CE
FTT7 Grp-17 ~650 CE
-BY3610 Grp-17a ~750 CE
--BY3101 Grp-17a1 - Ola Johnson Viktil^ b. 1749 Trondelag, Norway
--Y63570 Grp-17a2 - Byrne^ (2)
-Y72806 Grp-17b ~1000 CE - Clark, Crowley
--FT418455
---FTA37584 - Coyle
-BY182595/Y159063^ Grp-17c ~1400 CE
--BY69496 -
---BY202118 - White^
-Y31615 Grp-17d ~700 CE - McGinnis* (4), Guinnessy
--BY48168 Grp-17d1 ~1000 CE - O'Neill (2)
---Y31616 ~1100 CE - Ross*.
----Y92243 ~1200 CE - R. Allen, Henretta (v. of Hanratty), Coghlan*, Mcgeary*
----FT435332 ~1300 CE - Callihan (Me), O'Neill, Callahan (2)**
----Y40418 ~1200 CE
-----BY63950 - McGinnis, Humphries (G. Mac Unfraidh)
-----Y38657
------FT36397 - McConville (v. of McConnell), Ross
------BY3809 - Arvin, Irvin
---FT333432 ~1200 CE - McAvoy*
----FT333961 - McEvoy, McVey
----FTA46313 - McEvoy (2)
* Y-25/GD1or2 match
** Y-37/GD4 match
^ YFull SNP match
Others: Big Y (700) match
Ross (the only clearly Scottish surname here) is likely due to a McConville N.P.E.