Clean up is going well, added a few more classes to group like operations. For example, there is now a DB class for the interactions around the structure of the DB. Ran into a bit of a problem with using namespace global variables that I ended up overcoming by creating a couple of new objects, one of the objects now contains the NNTP details which were global variables and now get passed around to different methods that require them.
During the DB clean up I stumbled upon a neat little feature of LOAD INFILE which was if the data in a file is tab separated that it automatically separates the data into columns which saved me some cojiggery. I have also found out that parameterization of the ODA class statements for MySQLConnection doesn’t work for anything EXCEPT for parameters used within ()’s. This was a bit of a set back as I had hoped to stop doing string manipulation for base group names but unfortunately I will have to keep doing that at least for the time being.
The final item that I am looking at is the difference between doing comparisons in MySQL vs List<string> comparisons. Without any cache of the operation it takes around 40 seconds to do a distinct comparison and insert for 300k records, I can only guess that this is going to be a VERY costly way to generating a release list so I will be doing two List<strings> and comparing them to see how much better than performs. My thoughts are not so much.
A forgotten accomplishment was that I started the initial DB structure creation methods in the application instead of notepad.