1
Vote

MEMORY LEAK - Interactive window Python 2.7 (PTVS 2.0, VS212 Ultimate)

description

Hi,

I installed PTVS 2.0 VS2012, but when creating any python project or simply opening Python 2.7 interactive, memory of the process devenv.exe goes >3GB, until VS crashes.

Can this be a problem related to the setting of some enviromental variables on my machine?

What can be the cause of this HUGE memory leak?

I would appreciate some help because I would like to use PTVS!

thanks

file attachments

comments

A_quant wrote Aug 8 at 2:15 PM

This is my environment. + DB to follow

A_quant wrote Aug 8 at 2:16 PM

Python environment

A_quant wrote Aug 8 at 2:17 PM

DB zip (+ 3 parts to follow)

A_quant wrote Aug 8 at 2:18 PM

part1

A_quant wrote Aug 8 at 2:19 PM

part2

A_quant wrote Aug 8 at 2:19 PM

part3

A_quant wrote Aug 8 at 2:21 PM

Hi,

can you please check the completionDB generated by your tool.


many thanks

Zooba wrote Aug 8 at 4:40 PM

Thanks for uploading this. However, something seems to have become corrupted in the split ZIP file and I'm not able to extract all the files (I got 149 out of 5382). Perhaps you could roughly split the files in half and email two ZIP files? 5MB should be well below the attachment limit (I think our incoming limit is 10MB, but I'm not 100% certain).

A_quant wrote Aug 8 at 5:01 PM

Hi
There are 4 attachments

DB.ZIP
DB.z01
DB.z02
DB.z03

if you put all 4 in a folder and you just click on DB.zip i will do the job.

A_quant wrote Aug 8 at 5:02 PM

  • it will do the job

A_quant wrote Aug 8 at 5:08 PM

let me know if you still have issues when unzipping files

Zooba wrote Aug 8 at 5:23 PM

That's what I did. 7-zip can't handle it at all, Windows Explorer and Python's zipfile module can show me the full list of files but can't extract them all because one of the files is corrupted.

A_quant wrote Aug 8 at 5:52 PM

zip

A_quant wrote Aug 8 at 5:56 PM

zip new

A_quant wrote Aug 8 at 6:05 PM

z01 new

A_quant wrote Aug 8 at 6:11 PM

new z02



please use just the last 3 files

thanks!

Ptools wrote Aug 8 at 6:19 PM

you can also use onedrive or google drive to easily send large files...

Zooba wrote Aug 8 at 6:31 PM

I'm still getting exactly the same behaviour with the new files. Using an online service like OneDrive or DropBox with a single ZIP file would probably be simpler.

A_quant wrote Aug 8 at 6:53 PM

ok. I just put them in zip folders manually. please consider the last 3 files

A_quant wrote Aug 8 at 6:54 PM

part2

A_quant wrote Aug 8 at 6:55 PM

part3

Zooba wrote Aug 8 at 8:21 PM

Thanks, that worked fine. There doesn't seem to be anything obviously wrong with the DB itself, but I've got some other things I can try to see where the problem is coming from.

A_quant wrote Aug 8 at 8:59 PM

Hi,

I would really appreciate any hint to a possible cause!


Many thanks!

Zooba wrote Aug 8 at 9:40 PM

I'm not seeing any memory problems when I use that database with PTVS 2.0 or our latest versions.

Have you sent the information from Tools-Python Tools-Diagnostic Info... previously? I can't find it right now. If you can get it again while the memory is rising then there'll be a few entries near the top that could be very helpful.

I can produce a similar memory problem by adding C:\Python27\Lib as a search path in my project, and this is equivalent to having around one thousand files in the project (I think I mentioned earlier that we normally treat Lib and site-packages specially, but adding them as a search path bypasses this and causes problems) - you'll also see high CPU usage if this is the issue.

However, since your problem also occurs on an empty project and we ignore the global value of PYTHONPATH (on 2.0 at least) I'm not sure how we could be picking up these extra files. One possibility is that your empty project is in a folder with lots of other .py files, but again, it seems unlikely. VS 2012 on Windows 7 may also be related, though I don't know why or how. It'll take me a little while to get that environment set up but I'll try it.

Sending the diagnostic info from PTVS while memory usage is growing may hold the answer here. It'd be great if you can email that through (or post it here, but be aware that it'll have user names/paths in it).

A_quant wrote Aug 9 at 9:40 AM

Hi,

thanks for the reply.
I am using PTVS 2.1 Beta 2 right now and the issue is identical to PTVS 2.0.

Are there other Enviromental Variables settings that may trigger this issue?

PLease keep in mind that the memory issue is strictly and only connected to the completionDB. If I delete it it works fine. but clearly I am unable to exploit the benefits of your tool.

it looks like the intellisense struggles to read efficiently the database. I am using VS 2012 Ultimate.

I was not planning to debug this myself but I will start to do it if necessary because I would like to mixed debug python in VS. I will look at devenv.exe using VMMAP and track down its memory usage.

A_quant wrote Aug 9 at 11:55 AM

it can be also a compatibility issue of VS2012 and Windows7. I'll try with

http://www.microsoft.com/en-us/download/confirmation.aspx?id=36020

A_quant wrote Aug 9 at 4:41 PM

hi

I started to look into the managed using VMMap.

The managed heap is out of control. Objects just pile up and especially in the case of "refresh completion database", the very same ones. memory gets released only when system is in low memory situation.

please have a look at

http://stackoverflow.com/questions/2957105/how-can-you-explore-the-managed-heap-in-a-net-application-to-identify-possible

and

http://msdn.microsoft.com/en-us/magazine/cc534993.aspx

But I am sure you have plenty of other tools to profile and debug objects in the heap.

can you please profile the heap once hitting the completion DB. just compare new objects with previous ones and trigger gc if necessary before the system goes out of memory.

Many thanks!

Zooba wrote Aug 9 at 5:32 PM

I checked in a fix during the week for the repeatedly-clicking-refresh issue, so that will be in our next release. However, I haven't been able to reproduce the problem where memory fills up without the user doing anything apart from opening the interactive window or a new project. Those may be due to an environment variable, but I don't know which one it may be.

Can you send the output of running set at a command prompt so I can have a look on Monday at what the possibilities are?

A_quant wrote Sep 2 at 4:24 PM

Hi

I am currently use PTVS 2.1 VS 2012.msi and the memory management of the CompletionDB is much improved.

However the issue is not solved yet.

I observe the following now:

1) I load up a large Python Solution.

2) The CompletionDB is up to date

3) the memory of devenv.exe goes up to 3GB and then gets dumped, however it does it in a progressive decreasing way, first to 1.6gb, 2gb, 2.2gb,.... up to when no memory is dumped and VS crashes.

Question: Why now I observe this behaviour of the Garbage collector? Does this behaviour reflect the changes you made to address the issues with the completionDB lookup?


Many thanks!

A_quant wrote Sep 2 at 4:40 PM

UNHANDLED EXCEPTIONS FROM PROCESS 52060:

02/09/2014 16:28:36
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.String.CtorCharCount(Char c, Int32 count)
at System.Char.ToString()
at System.String.Concat(Object arg0, Object arg1, Object arg2)
at System.IO.Path.InternalCombine(String path1, String path2)
at System.IO.FileSystemEnumerableIterator1.CreateSearchResult(SearchData localSearchData, WIN32_FIND_DATA findData)
at System.IO.FileSystemEnumerableIterator
1.MoveNext()
at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection)
at Microsoft.PythonTools.Interpreter.PythonInterpreterFactoryWithDatabase.MakeTypeDatabase(String databasePath, Boolean includeSitePackages)
at Microsoft.PythonTools.Interpreter.PythonInterpreterFactoryWithDatabase.GetCurrentDatabase()
at Microsoft.PythonTools.Interpreter.Default.CPythonInterpreter.<OnNewDatabaseAvailable>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.<ThrowAsync>b__1(Object state)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

Zooba wrote Sep 2 at 6:19 PM

Interesting. It looks like we're loading the database again, which doesn't really make sense since it should already be loaded.

Do you have any tools running that may be modifying files in your DB? If you aren't repeatedly hitting Refresh DB, there must be something else that keeps modifying our files and causing us to reload.

A_quant wrote Sep 2 at 7:31 PM

Hello :)

thanks for the reply. No, I dont think so. I don't have any tools that can be related to intellisense or Debug Symbols or modify DB content

Is it possible to just enforce the database to be static?

A_quant wrote Sep 3 at 7:37 PM

Hi,

Any idea about what can trigger the DB to be constantly changed?

thanks

Zooba wrote Sep 3 at 9:04 PM

Sorry. I've been trying to look into this, but we have a lot of distractions going on right now and I haven't had as much time as I'd have liked.

Re-reading your earlier post, I see that you've opened up a large solution. It's entirely likely that there is some code in that solution that our analyzer doesn't handle well. Here's something you can try:
  1. Close VS completely.
  2. Create a registry key at HKEY_CURRENT_USER\Software\Microsoft\PythonTools\12.0\Analysis\Project
  3. Create a DWORD value called CallDepth in this key and set its value to zero.
  4. Reopen VS and the large solution.
This will significantly reduce the intelligence of our code completions, but it may keep the memory usage under control. If you don't see any change in the memory at all, there is another bug somewhere that I don't think we'll be able to find without access to your code (as we've discussed before, I believe).

If, as we've discussed earlier, you are seeing the same memory errors even without opening a project, then I have absolutely no idea how we could be responsible for that. PTVS does not load databases or do any analysis until needed.

Zooba wrote Sep 3 at 9:06 PM

Oh, one other option that might be helpful (though it seems unlikely). We have an "Auto-analyze standard library" option under Tools->Options->Python Tools->General. If you disable this, it may prevent some triggering.

I think it's unlikely to help because if you're not actually doing anything then this trigger won't do anything. It should only go when you create a new project or change the active environment for a project.

Zooba wrote Sep 3 at 9:07 PM

Fixed instructions from above (just saw that you're on VS 2012, not VS 2013):
  1. Close VS completely.
  2. Create a registry key at HKEY_CURRENT_USER\Software\Microsoft\PythonTools\11.0\Analysis\Project (note 11.0 == VS 2012)
  3. Create a DWORD value called CallDepth in this key and set its value to zero.
  4. Reopen VS and the large solution.

A_quant wrote Sep 4 at 12:32 PM

Hi,

thanks for the reply.

I cannot see this options "We have an "Auto-analyze standard library" option under Tools->Options->Python Tools->General." and I am using PTVS 2.1

I created the registry key as above and it got worse. memory just shoots up to 3.2GB

Furthermore I could only see the key HKEY_CURRENT_USER\Software\Microsoft\PythonTools

I had to create \11.0\Analysis\Project

Are there any default REGISTRY KEY I should be able to see by default?


Please see below some exceptions thrown related to PythonTools

A_quant wrote Sep 4 at 12:34 PM

UNHANDLED EXCEPTIONS FROM PROCESS 11068:

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
at System.IO.Path.GetFullPathInternal(String path)
at System.IO.File.InternalExistsHelper(String path, Boolean checkHost)
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClass6.<EnqueueFile>b__5()
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClassc.<EnqueWorker>b__b(Object dummy)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.String.CtorCharPtrStartLength(Char* ptr, Int32 startIndex, Int32 length)
at System.IO.PathHelper.ToString()
at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClass6.<EnqueueFile>b__5()
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClassc.<EnqueWorker>b__b(Object dummy)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
at System.IO.Path.GetFullPathInternal(String path)
at System.IO.File.InternalExistsHelper(String path, Boolean checkHost)
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClass6.<EnqueueFile>b__5()
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClassc.<EnqueWorker>b__b(Object dummy)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClass6.<EnqueueFile>b__5()
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClassc.<EnqueWorker>b__b(Object dummy)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
at System.IO.Path.GetFullPathInternal(String path)
at System.IO.File.InternalExistsHelper(String path, Boolean checkHost)
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClass6.<EnqueueFile>b__5()
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClassc.<EnqueWorker>b__b(Object dummy)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
at System.IO.Path.GetFullPathInternal(String path)
at System.IO.File.InternalExistsHelper(String path, Boolean checkHost)
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClass6.<EnqueueFile>b__5()
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClassc.<EnqueWorker>b__b(Object dummy)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
at System.IO.Path.GetFullPathInternal(String path)
at System.IO.File.InternalExistsHelper(String path, Boolean checkHost)
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClass6.<EnqueueFile>b__5()
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClassc.<EnqueWorker>b__b(Object dummy)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
at System.IO.Path.GetFullPathInternal(String path)
at System.IO.File.InternalExistsHelper(String path, Boolean checkHost)
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClass6.<EnqueueFile>b__5()
at Microsoft.PythonTools.Intellisense.ParseQueue.<>c__DisplayClassc.<EnqueWorker>b__b(Object dummy)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.String.CtorCharPtrStartLength(Char* ptr, Int32 startIndex, Int32 length)
at System.IO.FileSystemWatcher.CompletionStatusChanged(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* overlappedPointer)

at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

03/09/2014 19:42:17
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Threading.Threa

A_quant wrote Sep 4 at 12:36 PM

please see the latest attached file!

Thanks!

Zooba wrote Sep 4 at 5:10 PM

Sorry, had to double check the name. It's called "Automatically generate completion DB on first use"

You shouldn't see any existing registry keys - we're deep into unsupported settings territory and have almost never used these before (except for testing). We added them so we could help people in this situation keep working. Not going well so far :(

Unfortunately, those exceptions aren't really helpful. OutOfMemoryException can be thrown at any time when memory is not available and so the stack traces don't tell us much. The one thing they do show is that we are trying to analyze your project.

What this probably means is that your projects are too big for PTVS to handle. Changing the setting I suggested before shows that it probably isn't code complexity and is probably just the number of files you have in your loaded projects. If you have less projects loaded in VS then you should be able to work fine under the default settings.

We have done testing up to 10,000 files in a single project, and a few tests with 10-100,000 files spread across projects, but these have not been "real" applications and so you've got a more realistic case.

Approximately how many files do you have in all your projects? And roughly how many projects? It'd be good for us to get a perception of just how big of a "real" solution we can handle. I think we're probably trying to load every Python file into memory all at once, so if you have 2-3GB worth of code then that'll be the problem. It would be great if you could help us understand your overall scenario better, as we simply haven't encountered anyone trying to do this much in the past.

The two immediate fixes you have are:
  • multiple solutions with fewer projects in each
  • put shared libraries into the site-packages folder
(Right now Search Paths are treated like the files are part of the project, so they won't help you, but the site-packages folder in a Python install or a virtual environment is treated specially and won't have the same memory issues. I'm aware of issues causing multiple projects and Search Paths to use too much memory, and I have plans to fix those when I have time available, but we have a lot of work on our plate and I simply haven't been able to make such sweeping changes for 2.1.)

A_quant wrote Sep 4 at 6:20 PM

Hi

thanks for your reply.

Solution contains ~80 projects and 15K files, 1.2GB of code.

Zooba wrote Sep 4 at 7:36 PM

Yeah, that's probably blowing past what we can handle :)

Are you able to split it up into smaller groups of projects? That may not make sense for what you're working on, but it's probably the only way you'll be able to get it to load in 2.1...

A_quant wrote Sep 4 at 7:39 PM

nope :)

using PyDev

Ptools wrote Sep 4 at 7:58 PM

a_quant - are all those projects python or is it a mix of languages?

A_quant wrote Sep 4 at 8:11 PM

python

A_quant wrote Sep 5 at 11:46 AM

I hope you guys can improve PTVS for large projects. It would be great to use VS as an IDE for python

Zooba wrote Sep 5 at 5:02 PM

Yep, we're certainly looking into it. Unfortunately, right now our architecture simply can't handle that much source code (it's more than VS can physically handle), so until we get enough time to revise that it's going to remain a fundamental limitation.

We're also looking into some options to simply disable IntelliSense completely, which would still let you use debugging, but you wouldn't see code completions or tooltips while editing. We may be able to do that for 2.1 as an advanced/secret option - would you be interested?

A_quant wrote Sep 5 at 5:32 PM

not sure. When I write code I just need a good intellisense (visual assist is a fantastic tool). usually I dont need to debug the code once I write it :)

Ptools wrote Sep 5 at 5:48 PM

well, specifically, the option would turn of intellisense for selected projects which you choose. do you need intellisense for all 80 projects at the same time, or can some be selectively turned off temporarily?

Zooba wrote Sep 5 at 6:43 PM

To be clear, with that option you'd have to manually go through each project and add some special setting to disable analysis. This would stop us from ever looking at the files in those projects, so you'd get nothing within in and any other projects that were still being used would not be able to see completions for classes/etc. in these projects. (Note that this is practically identical to manually unloading each project today.)

It's also considerably easier for us to create one "kill switch", so whether we can add a reliable and deeper level of customization will depend heavily on our schedule.

But it sounds like the intellisense is the most important feature to you, which is what I suspected. Unfortunately, until we can rearchitect our system, the best option available to you is still to unload the projects that you don't need.

A_quant wrote Sep 5 at 7:06 PM

No, I dont need intellisense for all the projects, that would be a sensitive temporary solution