PDA

View Full Version : Living DNA Raw Data - Now Available



ajc347
10-06-17, 23:02
Living DNA Raw Data files are now available to download from their site.

I downloaded mine earlier and must admit to being somewhat disappointed with it.

The Y-SNP data appears to only show the SNP's I tested positive for and doesn't list the other SNP's they tested. This renders the data useless for aiding further targeted testing elsewhere (e.g. YSEQ).

My mtDNA data consisted of 9 markers drawn from the HVR2 and Coding Region, and the autosomal data comes in a vcf.txt format and isn't uploadable to either GEDmatch or My Heritage.

Overall, I'm pretty unimpressed. :sad-2:

Promenade
11-06-17, 02:26
Thanks for letting everyone know. I was excited to see that it was finally available, but you're right I'm fairly unimpressed as well. For people who have only used LivingDNA, not having the option to use GEDmatch will be a huge disappointment.

Maciamo
11-06-17, 08:10
I have discussed this with Living DNA and they assured me that they are working on making the raw data compatible with GEDMatch and other websites that analyse health risks. I suppose it is just a beta version for the time being.

Maciamo
11-06-17, 09:04
I have checked the raw data and there may be a way to make it compatible with GEDMatch by reformatting it to look like a 23andMe file. What I did is open a text editor and delete all these (use search and replace):

. . VC=SNV GT 0/0
. . VC=SNV GT 0/1
. . VC=SNV GT 1/0
. . VC=SNV GT 1/1

After that, you can remove the spaces between the alleles like that (for each combination)

search for 'G A' replace by 'GA'

This is easy enough. The problem is that 23andMe uses the following order:

rsid chromosome position genotype

While Living DNA uses that order:

chromosome position rsid genotype

So if we could find a way to move the rsid from third to first position that would make the file identical to the 23andMe layout. I will ask them if they can do it. If not, does anybody here have programming skills to create a file converter to move the position of the rsid? The command would be something like "cut rs* + tab" then "paste at the beginning of same line". It sounds easy enough, but unfortunately I don't have the programming skills to do it.


Dienekes used this script to standardise 23andMe and FTDNA genomes:


standardize <- function(genotypefile, company='23andMe')
{
if (company=='23andMe') {
X<-read.table(genotypefile)
}
if (company=='ftdna') {
X<-read.table(genotypefile, sep=',',skip=1)
}
write.table(X[order(X[,1]),],file='genotype.txt',quote=F,row.names=F,col.names =F)
}


This only removes the file headers, but it could be a variant of that using the same script in R.

Maciamo
11-06-17, 09:36
Actually it's not going to work like that. The allele mentioned are just the two possibilities. The actual results are the 0/1 at the end of the line. So this file format is not going to be usable at all.

ajc347
11-06-17, 12:13
It looks like GEDmatch will accept vcf files after all, via their new Genesis beta. I've just been able to upload my raw data there without problem.

Whilst there are no admixture tools available on Genesis as yet, I'm hopeful that they will be added in due course. http://www.anthrogenica.com/images/smilies1/smile.gif

Maciamo
12-06-17, 11:55
It looks like GEDmatch will accept vcf files after all, via their new Genesis beta. I've just been able to upload my raw data there without problem.

Whilst there are no admixture tools available on Genesis as yet, I'm hopeful that they will be added in due course. http://www.anthrogenica.com/images/smilies1/smile.gif

Good to know. But there are many other websites (e.g. DNA.Land) and programmes (like R to run Dodecad/Eurogenes admixtures manually) which do not seem to accept that format.

EDIT: I tried to upload the file through GEDMatch Genesis. It worked, but the file isn't usable in the regular GEDMatch. The site says that the two databases will eventually be merged (could be in months or years though) and in the meantime there is almost nothing of any use in Genesis. No admixture for example.

srdceleva
12-06-17, 13:07
Thanks for letting everyone know. I was excited to see that it was finally available, but you're right I'm fairly unimpressed as well. For people who have only used LivingDNA, not having the option to use GEDmatch will be a huge disappointment.
How long did it take you to get your results after the lab accepted your kit for testing? I was notified Last week that my kit is not in testing phase and my predicted completion time is in September....did it really take that long after the lab started for testing? Just to get to this point its already been over a month after I sent my kit back...

Sent from my KIW-L21 using Tapatalk

Promenade
12-06-17, 14:14
How long did it take you to get your results after the lab accepted your kit for testing? I was notified Last week that my kit is not in testing phase and my predicted completion time is in September....did it really take that long after the lab started for testing? Just to get to this point its already been over a month after I sent my kit back...

Sent from my KIW-L21 using Tapatalk

It took them 12 days for the lab to receive it, another 18 days for the lab to start testing on it and then a little over a month and a half for results

I1a3_Young
12-06-17, 14:32
Actually it's not going to work like that. The allele mentioned are just the two possibilities. The actual results are the 0/1 at the end of the line. So this file format is not going to be usable at all.

Do the 0/1 correspond to A/T directly or are the binary results reflecting SNP positive/negative?

Maciamo
12-06-17, 16:04
Do the 0/1 correspond to A/T directly or are the binary results reflecting SNP positive/negative?

Each line mentions the ancestral and derived allele. The 0/1 shows the number of copies for the derived allele (1/1 means one inherited from each parent). So it's not impossible to convert the file. A bit tricky though. That would necessitate to create a script that read the ancestral and derived allele, then replaces the 0 and 1 accordingly for each SNP.

Maciamo
12-06-17, 16:07
How long did it take you to get your results after the lab accepted your kit for testing? I was notified Last week that my kit is not in testing phase and my predicted completion time is in September....did it really take that long after the lab started for testing? Just to get to this point its already been over a month after I sent my kit back...


From what I hear the process is getting faster. It used to be 4 months, then 2 and a half, and now the latest kits have a predicted completion time of only about a month.

srdceleva
12-06-17, 18:09
From what I hear the process is getting faster. It used to be 4 months, then 2 and a half, and now the latest kits have a predicted completion time of only about a month.
Great thanks for the info, hopefully I'll get to post the results soon.

I1a3_Young
12-06-17, 20:27
Each line mentions the ancestral and derived allele. The 0/1 shows the number of copies for the derived allele (1/1 means one inherited from each parent). So it's not impossible to convert the file. A bit tricky though. That would necessitate to create a script that read the ancestral and derived allele, then replaces the 0 and 1 accordingly for each SNP.

Vlookup() in Microsoft Excel could probably do that, as well as putting the columns in the correct order. I don't know how to automate it though.

Maciamo
12-06-17, 20:34
I just found this site (https://samtools.github.io/bcftools/howtos/index.html) and that one (https://github.com/2sh/vcf-to-23andme) which have scripts to convert VCF files to/from 23andMe format. It could be the solution. It looks a bit technical though.

There is also this site (http://www.math.mun.ca/~dapike/FF23utils/BigYvcf2csv.php) that instantly converts a (Big Y) VCF into FTDNA CSV format. I tried it and it does convert the 0/1 correctly into nucleobases, but the rsid and chromosome numbers are lost in the process. Only the position remains.

Maciamo
15-06-17, 13:16
I contacted Living DNA about this and they will release the raw data in 23andMe compatible format on 10th July.

srdceleva
15-06-17, 14:29
I contacted Living DNA about this and they will release the raw data in 23andMe compatible format on 10th July.
Fantastic, im amazed how responsive living DNA is to feedback.

Sent from my KIW-L21 using Tapatalk

Johane Derite
10-07-17, 16:40
I contacted Living DNA about this and they will release the raw data in 23andMe compatible format on 10th July.

Did the raw data option come out? Anyone on Livingdna have anyfeedback?

Dibran
10-07-17, 18:18
I contacted Living DNA about this and they will release the raw data in 23andMe compatible format on 10th July.


Mine still shows as 1kb vsc. Is there a time table when this will go live? for everyone? or is it gradual?

Twilight
11-07-17, 09:08
Living DNA Raw Data files are now available to download from their site.

I downloaded mine earlier and must admit to being somewhat disappointed with it.

The Y-SNP data appears to only show the SNP's I tested positive for and doesn't list the other SNP's they tested. This renders the data useless for aiding further targeted testing elsewhere (e.g. YSEQ).

My mtDNA data consisted of 9 markers drawn from the HVR2 and Coding Region, and the autosomal data comes in a vcf.txt format and isn't uploadable to either GEDmatch or My Heritage.

Overall, I'm pretty unimpressed. :sad-2:
I'm pretty sure LivingDNA is young enough to accept feedback easily. For DNA components outside of the British Isles, LivingDNA is still at the Beta stage. ;) My LivingDNA results are not expected to be accessible to me until September 12th so I can't access my Raw Data yet, however I'll take a look at it when the time arises.

I1a3_Young
11-07-17, 18:50
GEDmatch now has something called Gedmatch Genesis will will accept LivingDNA files.

As for negative SNPs - you can pull that from the raw data if you are good with data manipulation. I will probably convert mine to 23andMe format and use a YSNP generator. They can be individually tested at the ISOGG Y-browser if you are interested in specific positions, it won't take that long if you only need to check a few branches.

Johane Derite
12-07-17, 10:14
I'm thinking to buy the test in the next couple of days, and if livingdna isnt going to be putting out a 23andme compatible rawdna soon then its a deal breaker, maciamo said 10th july but from users response here it seems they haven't done that yet : ( also I have a feeling their reliability is not as good with shipping etc

Johane Derite
12-07-17, 12:01
After contacting livingdna ceo, he assured me that as of yesterday people are able to download a 23 and me compatible file. Anybody who has taken the test can confirm this?

Maciamo
13-07-17, 06:57
I confirm that the Living DNA raw data is now 23andMe compatible. It works on the regular GEDmatch (not just Genesis).

Johane Derite
13-07-17, 09:13
I confirm that the Living DNA raw data is now 23andMe compatible. It works on the regular GEDmatch (not just Genesis).

Awesome! Thanks for this Mac :D

srdceleva
13-07-17, 13:46
I confirm that the Living DNA raw data is now 23andMe compatible. It works on the regular GEDmatch (not just Genesis).
Great. Maciamo you mentioned before that you were going to help living DNA with better descriptions of their haplogroups. Any new news on that front?

Sennevini
13-07-17, 15:00
Does the tokenizing and batch processing work well with you at regular Gedmatch? It says maybe "too few SNPs". Several tools work, but I would also like to use one-to-many.

Sennevini
15-07-17, 16:19
Does the tokenizing and batch processing work well with you at regular Gedmatch? It says maybe "too few SNPs". Several tools work, but I would also like to use one-to-many.

The txt-file doesn't seem complete yet, they are probably working on that.

Sennevini
20-07-17, 23:49
The txt-file doesn't seem complete yet, they are probably working on that.

correction: the autosomal raw file does indeed seem to contain 600k SNPs. It is just the batch-processing at regular gedmatch that doesn't work properly, which does only make the one-to-many comparison not possible. It does work at genesis gedmatch; likely later this year these databases will be merged.

Johane Derite
21-07-17, 00:02
correction: the autosomal raw file does indeed seem to contain 600k SNPs

Thanks for this update.


It is just the batch-processing at regular gedmatch that doesn't work properly, which does only make the one-to-many comparison not possible.

Sorry I didn't really understand this bit especially: " which does only make the one-to-many comparison not possible"

Sennevini
21-07-17, 10:56
Thanks for this update.


Sorry I didn't really understand this bit especially: " which does only make the one-to-many comparison not possible"

It seems that gedmatch (the original, not the genesis) doesn't batch process the LivingDNA files completely somehow; apparently you will therefore not see a list of potential matches (one-to-many list), which makes it difficult to point out your matches. You can use the one-to-one matching tool, but then you have to know your potential matches already, and you can use all kinds of admixture tools. Within the beta version genesis gedmatch, you can see a one-to-many list of potential matches, but genesis has fewer participants than regular gedmatch. That's how I understand it. Later this year problems might be solved.

Johane Derite
21-07-17, 10:57
That's how I understand it. Later this year problems might be solved.

Thanks sennevini

glashouse
24-07-17, 05:42
Hi, I just got living DNA results this week (submitted 24 April 2017) and I too can confirm it is GEDmatch compatible. I just created GEDmatch account and within 5 minutes I had livingDNA data uploaded without error.I chose the generic data option. Ron

Genomapp
02-11-17, 16:26
Hello! We just wanted to let you know that LivingDNA raw data is compatible with Genomapp. You just have to select VCF format when you transfer your raw data file to the app to be analyzed for health reports. If you need more information, please let us know.