top of page

Closing in

Updated: Sep 26, 2019

After accepting a slightly inferior hashing function compared to what was initially recommended to me (speed wise it can be improved but I currently lack the requisite knowledge of C to implement the function) I completed the single processor version of Rappel and began on the parallel processing core version. I'm using MPI (Message Passing Interface) to add parallel function. Initially just sending the data to the processing cores was a challenge but this was overcome with some clever computational thinking. Instead of keeping reads separate and sending multiple small chunks of an array, just concatenate all the reads you want to send, send a second array with all the lengths of each individual read and you have all the information necessary to easily cut each read into kmers and perform hashing and matching.

After completing this stage I realized there was a logic flaw in how I was processing each read length upon arrival at the processing core. Not a major problem but it does require exactly correct logic or you start producing "fake" read sequences that never existed. Parallel functionality is coming along and I'm actually getting kmer matching locations that include the the number of the core processing the data, the read number, the kmer number, the MSA sequence number and the position in which the kmer was found. This is a pretty awesome result. The speed of the program is fantastic. The only hitch so far is that the static arrays that are assigned to hold local information by the processing cores can't be assigned more than 1 million bytes. Logically this requires dynamic memory allocation. However, I'm having trouble simply replacing the static allocation with a dynamic allocation. This is a major problem as in situation where each core has to process more than a million bytes (most of the time really), the program suffers a segmentation fault and crashes. This is the new hurdle I'm working on. We'll see if I can finish a working prototype in time and "spin up the Deathstar." Stay tuned!

5 views0 comments

Recent Posts

See All

Finishing up and what I've learned

As much progress as can be made on Rappel in a jam-packed two month period is complete. Ultimately I was unable to produce a prototype capable of running on the Sunway Taihulight supercomputer. This i

Trouble Hashing It Out

Recently development of my project has switched to the C programming language and that has seen development slow significantly. I've been programming in Python and Bash for the last few years and I ca

5 Days in Beijing

We finally made it to Beijing and it was quite an experience. Beijing makes the town of Wuxi (where I'm staying) look like a small town by comparison. There are so many people all over the place at al

bottom of page