Well no documentation got done however I did the initial testing with the release stats. I ran a test with 300 releases getting the stats.
The stats that I was gathering were, size, parts, and total parts. Using the DB for everything ended up costing me around 1 hour and 50ish minutes. I did the tests then with just gathering the initial objects from the DB then paring down in memory. I was able to get the 300 releases done in a little under 6 minutes.
Had some problems with the size calculation due to SOME releases being posted larger than 120gb, but I got that all fixed up.
So, looking at the immediate list of things to do, 3 is now pretty much all complete, 2 is pretty much all done but ill be moving onto another phase for that. 7 has been started but thats gonna be a work on progress.
I am considering a few different paths right now, toying with the thought of trying to setup a parallel processing of releases as I feel that we the 6 minutes could be cut down to around 3 or 4 minutes.
I have been thinking about the master thread, basically its just going to query a bunch of the tables to see what the current status is.