Decided to change things up a little and do more work on the filesystem over the database as MySQL seems quite slow for doing single inserts type of operations. I have parametertized the transactions to try and speed up some insert time however that doesn’t seem to matter too much.
The next test would be to create a stored procedure for these inserts and call that instead of using inserts.
For the remainder of the week I need to do a code cleanup as it looks like shit at the moment from all the development. I need to spare out some to NNTPzzz for some unused (but maybe used later) and I will be creating a new class in the name space for the DBINFILE operations.
I did test out creating a large table which contains a “complete” view of each header and that worked pretty well but I need to figure out how to tokenize an INFILE transactions so that that I can move from flatfile to specific columns.