Debugging a Pyramid web application with PVTS

Sep 12, 2013 at 11:56 AM
Hi,

I am starting to develop a web application with Pyramid, using Pyramid's pserve server. I wanted to be able debug it with PTVS, and found the method where you attach to an existing pserve process described in https://pytools.codeplex.com/workitem/1534 One problem here is that you can't debug code run at server startup, as it's already executed by the time you connect to the process.

Discussing this in a training course with Paul from Agendaless Consulting (Pyramid's developers), he suggested an alternative method used with some other IDEs. I adapted it for use with PTVS as follows:
  1. Load your project into Visual Studio, and configure it to run in your Pyramid virtual environment. Virtualenv is supported directly with PTVS 2.0, and for 1.5 I achieved the same effect by adding a new python interpreter to PTVS's options, pointing to the python.exe in the virtual environment.
  2. Add the file <your-virtualenv-path>\Scripts\pserve-script.py to your project. Set it as the startup file.
  3. In your project's settings, debug, script arguments, set "development.ini" (or whatever ini file you want pserve to use). I don't think --reload works well here, so I avoid that.
  4. Debug as normal with F5. I find you can hit a number of importError exceptions, but can continue past them to get the server started. Breakpoints in any of your python code should be hit as expected (although it doesn't always seem reliable for me).
It's been great being able to examine the request object when debugging views, modify variables to test the effect, etc. I hope this info might be some use to other Pyramid users.

Murray
Coordinator
Sep 13, 2013 at 9:31 PM
Hi Murray,

Thanks for the info! I had tried to get this working in the past, and I've tried again today, but I always hit this problem when I start debugging (start without debugging gets the server started just fine). Did you encounter this by any chance? I've set my script arguments and startup file the way you described.
Starting server in PID 3640.
TypeError: can only concatenate tuple (not "instancemethod") to tuple
Press any key to continue . . .
Thanks,
Hugues
Sep 20, 2013 at 9:42 AM
Hi Hugues,

I have seen problems like that, particularly when I was starting, but I can't recreate the problem now. I'd recommend you make sure you are definitely using the correct virtual env, and the python.exe from that virtual env for all your development work.

One area that has tripped me up is running setup.py from a Pyramid project created with pcreate. You run this script from the command line to integrate your project into the virtual env. Two things:
  1. Make sure your command line is activated for the virtual env before running setup.py. Run <your-virtualenv-path>\Scripts\activate.bat
  2. Run setup.py with "python setup.py develop", not just "setup.py develop". This way, the virtual env python.exe will be used, and not any other python.exe registered with windows. (I wonder if virtualenv's activate.bat could to a better job of that?)
Hope this help

Murray
Sep 30, 2013 at 2:27 PM
Hi Hugues,

My colleagues also saw the "TypeError: can only concatenate tuple (not "instancemethod") to tuple" issue. It seems to be an issue with the waitress library (we have v0.8.7 installed in our virtual environment). We have a hack to get round it.

Modify waitress's task.py (...pyramidEnv\Lib\site-packages\waitress-0.8.7-py2.7.egg\waitress\task.py) so that the line calling "self.start_new_thread" instead calls "thread.start_new_thread".

I don't have any idea why this fails when debugging with PTVS, or why the change makes it work. That change isn't necessary when you are running pserve from the command line, or when using Run rather than Debug from Visual Studio. Is there something odd about the way PTVS debug interacts with threads?

Murray
Coordinator
Sep 30, 2013 at 5:38 PM
I haven't looked at that code, but it is very likely. PTVS has to hijack a bunch of methods in the standard library to keep track of thread creation, and to install debug hooks on new threads so that they can handle BP and stepping - and thread.start_new_thread is one of those. If the library in question stashes away a reference to the original thread.start_new_thread (rather than going through the module every time), then, after the attach, it will still be using that original, and will therefore spawn threads that PTVS cannot debug. The debugger should still be handling such threads gracefully when it encounters them, and not raise any exceptions, so that's a bug - but other problems stemming from it would not be unexpected.
Nov 26, 2013 at 2:14 PM
Edited Nov 26, 2013 at 2:14 PM
Hi,
today I experienced the same message using PTVS. Yesterday I tried all using sample django app - worked great. Today I installed pyramid 1.5a2 using PTVS, changed to 'Standard Python launcher', changed startup script to pserve-script.py, added script argument. After getting the error message, I googled and found this thread.

Even overriding the interpreter path and arguments to eventually just run:
pserve development.ini
Running from console works as expected
(env) C:\Users\sascha\Documents\Visual Studio 2013\Projects\PyramidTestProject1\
PyramidTestProject1>.\env\Scripts\pserve.exe .\env\myproject\development.ini
Starting server in PID 11100.
serving on http://0.0.0.0:6543
Running the same with PTVS by setting startup script or even overriding interpreter results in the message above or the debugger window just closes.

Ususally I use Eclipse/pydev on Ubuntu Linux to develop pyramid application. Debugging the same setup (zc.buildout or virtualenv/pyramid) is what I do many times as day. This is working with pydev since years. I would like to use PTVS for Azure deployment. Tried a year ago and stopped. Will spend some time to see where you guys are now?

Anyone interested in my runtime environment / PTVS diagnostic info or any other questions?
Coordinator
Nov 26, 2013 at 5:29 PM
Can I suggest holding off on trying PTVS again for a few months? We're leading up to releasing 2.1 which is focused entirely on improving web app support, and Pyramid is on the list.

We should have an alpha available by February next year, and we will have reasonably regular dev releases until then, so once we start putting some Pyramid-focused support in there it would be great to hear about your experiences with it.

Right now, we know we're lacking in support a bit - hence the specific focus for our next release. It does sound a bit like Pyramid (or rather waitress) is doing something strange when it comes to threading, but I'll be coming to look at that quite soon. Anything you manage to find out about making it work now would be appreciated, but don't be surprised if PTVS 2.0 has trouble.
Nov 27, 2013 at 4:59 PM
Thanks for giving insight into your roadmap targetting web app support, especially pyramid. Yesterday I noticed "Flask" & "Bottle" available via Azure web site gallery. Given this I am pretty sure you guys will offer pyramid support very soon.