1
Vote

MEMORY LEAK - Interactive window Python 2.7 (PTVS 2.0, VS212 Ultimate)

description

Hi,

I installed PTVS 2.0 VS2012, but when creating any python project or simply opening Python 2.7 interactive, memory of the process devenv.exe goes >3GB, until VS crashes.

Can this be a problem related to the setting of some enviromental variables on my machine?

What can be the cause of this HUGE memory leak?

I would appreciate some help because I would like to use PTVS!

thanks

file attachments

comments

A_quant wrote Aug 8 at 1:15 PM

This is my environment. + DB to follow

A_quant wrote Aug 8 at 1:16 PM

Python environment

A_quant wrote Aug 8 at 1:17 PM

DB zip (+ 3 parts to follow)

A_quant wrote Aug 8 at 1:18 PM

part1

A_quant wrote Aug 8 at 1:19 PM

part2

A_quant wrote Aug 8 at 1:19 PM

part3

A_quant wrote Aug 8 at 1:21 PM

Hi,

can you please check the completionDB generated by your tool.


many thanks

Zooba wrote Aug 8 at 3:40 PM

Thanks for uploading this. However, something seems to have become corrupted in the split ZIP file and I'm not able to extract all the files (I got 149 out of 5382). Perhaps you could roughly split the files in half and email two ZIP files? 5MB should be well below the attachment limit (I think our incoming limit is 10MB, but I'm not 100% certain).

A_quant wrote Aug 8 at 4:01 PM

Hi
There are 4 attachments

DB.ZIP
DB.z01
DB.z02
DB.z03

if you put all 4 in a folder and you just click on DB.zip i will do the job.

A_quant wrote Aug 8 at 4:02 PM

  • it will do the job

A_quant wrote Aug 8 at 4:08 PM

let me know if you still have issues when unzipping files

Zooba wrote Aug 8 at 4:23 PM

That's what I did. 7-zip can't handle it at all, Windows Explorer and Python's zipfile module can show me the full list of files but can't extract them all because one of the files is corrupted.

A_quant wrote Aug 8 at 4:52 PM

zip

A_quant wrote Aug 8 at 4:56 PM

zip new

A_quant wrote Aug 8 at 5:05 PM

z01 new

A_quant wrote Aug 8 at 5:11 PM

new z02



please use just the last 3 files

thanks!

Ptools wrote Aug 8 at 5:19 PM

you can also use onedrive or google drive to easily send large files...

Zooba wrote Aug 8 at 5:31 PM

I'm still getting exactly the same behaviour with the new files. Using an online service like OneDrive or DropBox with a single ZIP file would probably be simpler.

A_quant wrote Aug 8 at 5:53 PM

ok. I just put them in zip folders manually. please consider the last 3 files

A_quant wrote Aug 8 at 5:54 PM

part2

A_quant wrote Aug 8 at 5:55 PM

part3

Zooba wrote Aug 8 at 7:21 PM

Thanks, that worked fine. There doesn't seem to be anything obviously wrong with the DB itself, but I've got some other things I can try to see where the problem is coming from.

A_quant wrote Aug 8 at 7:59 PM

Hi,

I would really appreciate any hint to a possible cause!


Many thanks!

Zooba wrote Aug 8 at 8:40 PM

I'm not seeing any memory problems when I use that database with PTVS 2.0 or our latest versions.

Have you sent the information from Tools-Python Tools-Diagnostic Info... previously? I can't find it right now. If you can get it again while the memory is rising then there'll be a few entries near the top that could be very helpful.

I can produce a similar memory problem by adding C:\Python27\Lib as a search path in my project, and this is equivalent to having around one thousand files in the project (I think I mentioned earlier that we normally treat Lib and site-packages specially, but adding them as a search path bypasses this and causes problems) - you'll also see high CPU usage if this is the issue.

However, since your problem also occurs on an empty project and we ignore the global value of PYTHONPATH (on 2.0 at least) I'm not sure how we could be picking up these extra files. One possibility is that your empty project is in a folder with lots of other .py files, but again, it seems unlikely. VS 2012 on Windows 7 may also be related, though I don't know why or how. It'll take me a little while to get that environment set up but I'll try it.

Sending the diagnostic info from PTVS while memory usage is growing may hold the answer here. It'd be great if you can email that through (or post it here, but be aware that it'll have user names/paths in it).

A_quant wrote Aug 9 at 8:40 AM

Hi,

thanks for the reply.
I am using PTVS 2.1 Beta 2 right now and the issue is identical to PTVS 2.0.

Are there other Enviromental Variables settings that may trigger this issue?

PLease keep in mind that the memory issue is strictly and only connected to the completionDB. If I delete it it works fine. but clearly I am unable to exploit the benefits of your tool.

it looks like the intellisense struggles to read efficiently the database. I am using VS 2012 Ultimate.

I was not planning to debug this myself but I will start to do it if necessary because I would like to mixed debug python in VS. I will look at devenv.exe using VMMAP and track down its memory usage.

A_quant wrote Aug 9 at 10:55 AM

it can be also a compatibility issue of VS2012 and Windows7. I'll try with

http://www.microsoft.com/en-us/download/confirmation.aspx?id=36020

A_quant wrote Aug 9 at 3:41 PM

hi

I started to look into the managed using VMMap.

The managed heap is out of control. Objects just pile up and especially in the case of "refresh completion database", the very same ones. memory gets released only when system is in low memory situation.

please have a look at

http://stackoverflow.com/questions/2957105/how-can-you-explore-the-managed-heap-in-a-net-application-to-identify-possible

and

http://msdn.microsoft.com/en-us/magazine/cc534993.aspx

But I am sure you have plenty of other tools to profile and debug objects in the heap.

can you please profile the heap once hitting the completion DB. just compare new objects with previous ones and trigger gc if necessary before the system goes out of memory.

Many thanks!

Zooba wrote Aug 9 at 4:32 PM

I checked in a fix during the week for the repeatedly-clicking-refresh issue, so that will be in our next release. However, I haven't been able to reproduce the problem where memory fills up without the user doing anything apart from opening the interactive window or a new project. Those may be due to an environment variable, but I don't know which one it may be.

Can you send the output of running set at a command prompt so I can have a look on Monday at what the possibilities are?