After accepting a slightly inferior hashing function compared to what was initially recommended to me (speed wise it can be improved but I currently lack the requisite knowledge of C to implement the function) I completed the single processor version of Rappel and began on the parallel processing core version. I'm using MPI (Message Passing Interface) to add parallel function. Initially just sending the data to the processing cores was a challenge but this was overcome with some clever computational thinking. Instead of keeping reads separate and sending multiple small chunks of an array, just concatenate all the reads you want to send, send a second array with all the lengths of each individual read and you have all the information necessary to easily cut each read into kmers and perform hashing and matching.
After completing this stage I realized there was a logic flaw in how I was processing each read length upon arrival at the processing core. Not a major problem but it does require exactly correct logic or you start producing "fake" read sequences that never existed. Parallel functionality is coming along and I'm actually getting kmer matching locations that include the the number of the core processing the data, the read number, the kmer number, the MSA sequence number and the position in which the kmer was found. This is a pretty awesome result. The speed of the program is fantastic. The only hitch so far is that the static arrays that are assigned to hold local information by the processing cores can't be assigned more than 1 million bytes. Logically this requires dynamic memory allocation. However, I'm having trouble simply replacing the static allocation with a dynamic allocation. This is a major problem as in situation where each core has to process more than a million bytes (most of the time really), the program suffers a segmentation fault and crashes. This is the new hurdle I'm working on. We'll see if I can finish a working prototype in time and "spin up the Deathstar." Stay tuned!
コメント