I presume that you made a DNA test with MyHeritage or FTDNA?,all people with only North African background get some 10% to 30% "South European" which is misleading, it's most likely a misinterpretation by FTDNA and MyHeritage, it's most likely ancient origin from antiquity,(Roman,Greeks,Vandals), or neolithic/mesolithic Europe.You can see it on Y haplogroup and mtDNA haplgoups distribution for North Africans (in Eupedia for exemple), They are mostly Eurasian haplogroup (70% to 75% for mtDNA haplogroup), North Africans males are carriers or North Africans haplogroups E-M81,E-L19,E-V65,some E-M123 lineages, (60 to 80%),middle easteren haplogroups J1 (10 to 25%,), subsaharan haplogroups (E-M2,A,B) 10% and European haplogroups (R-M269,C-V20,G-L140,R1a). (7 to 10%).
People with Ottoman or Andalousian or more recent roots usually know something about there Europeans origins by oral traditions.