Memory Leak

[robbjordan]

I am running a 64 bit machine with windows 10. I have 16gb ram.

I am converting a SQL Table that is ~ 40gb.

The program crashes everytime at ~ 9 gb.

You can watch the memory leak happen as the SQL Lite db is being created....there is a steady climb in memory until approximaly 80% ram is used...

The code needs to be refactored to manage this. Not sure of the solution as of yet.

[jschultz]

I'm afraid I'm not using this code any longer since I discovered sqlalchemy allows you to write fast and friendly Python scripts to access MSSQL (plus SQLite, MySQL, PostgreSQL, etc.) databases using fast and friendly Python scripts.

But I presume there are memory leak detection tools along the lines of valgrind/memcheck that could help?

[robbjordan]

I guess not too many using this for larger systems. I ended up writing a different solution to take the output files, merge them..and use the sqllite3 command line to do the migration. Pretty simple and required really no code and did the 40gb in a few hours.