Support python in cygwin

Mar 1, 2015 at 5:26 AM
Hi PTVS folks,

I had difficulty in using PTVS with python in cygwin. Is it supported?

Thanks.
Coordinator
Mar 1, 2015 at 5:55 AM
I don't think we have actually ever tried that. You'll likely have problems with it because PTVS expects the layout of a Python distro to be that typical for a Windows install (i.e. all in a single folder, with libraries under Lib) by default, and in Cygwin, I assume, it replicates a Unix-like layout? Still, it should be possible to register it as a custom interpreter, and specify the library path manually.

What kinds of issues are you running into?
Mar 1, 2015 at 6:38 AM
Hi pminaev,

Here is a concrete issue I ran into. I installed numpy on cygwin python 2.7.
  1. In cygwin window, I do not have issue in "import numpy".
  2. In the PSTV's interactive window, however i am getting:
import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/numpy-1.9.1-py2.7-cygwin-1.7.34-x86_64.egg/numpy/init.py", line 170, in <module>
from . import add_newdocs
File "/usr/lib/python2.7/site-packages/numpy-1.9.1-py2.7-cygwin-1.7.34-x86_64.egg/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/usr/lib/python2.7/site-packages/numpy-1.9.1-py2.7-cygwin-1.7.34-x86_64.egg/numpy/lib/init.py", line 17, in <module>
from . import scimath as emath
ImportError: cannot import name scimath

I checked the file scimath is under the folder /usr/lib/python2.7/site-packages/numpy-1.9.1-py2.7-cygwin-1.7.34-x86_64.egg/numpy/lib/ "from cygwin"'s perspective. (The same thing with slightly different error happens with numpy1.7 from cygwin distribution.)

Not sure about the root issue. It would be great if PTVS is supporting the tool-chain in cygwin which I find very useful for linux/windows hybrids. :)

thanks.
Coordinator
Mar 1, 2015 at 8:22 AM
By "Cygwin window", do you mean their Bash terminal? I suspect that they might be setting PYTHONPATH there in some way, and it's not taken care of when running python.exe directly.

If you run python.exe from the Windows command prompt, do you get the same error?
Mar 1, 2015 at 6:43 PM
Got the same error in powershell and cmd.

Checked the sys.path in both cygwin window and PTVS. Does not see any obvious difference there.
Coordinator
Mar 3, 2015 at 5:49 PM
Turns out this is actually a generic PATH issue, not PYTHONPATH. Basically, when you use the Cygwin shortcut to open the terminal window, they add things like /usr/bin and /usr/local/bin to PATH for you; but when you run the binaries directly, as PTVS does, you're responsible for doing that. In particular, the following made it work for me:
os.environ['PATH'] = '/usr/local/bin:/usr/bin:/usr/lib/lapack:' + os.environ['PATH']
import numpy
(I was getting a slightly different error though, which didn't mention scimath but rather lapack; it's possible that there is more stuff that needs to be added in your case. Just print out $PATH from Cygwin shell and grep for /usr in it, and copy all the entries.)

Now as to how to avoid having to do this in the code. In PTVS 2.1, we didn't have a feature to let you edit environment variables for projects, so the only way it would work there is by adding all that /usr/... stuff to the global PATH environment variable (and living with the fact that for non-Cygwin programs, it is an invalid entry). However, the feature was implemented for 2.2, so if you get the most recent dev build, you should be able to just define the PATH accordingly in project properties.

Note also that there are other things that don't work right now. In particular, debugger doesn't work because it expects to see msvcrt module on Windows. You can avoid that error by going to Tools -> Options -> Python Tools -> Debugging, and unchecking both "Wait for input ..." checkboxes, but going forward, we should do better there and fall back to an alternative way of doing this if msvcrt is not available.
Coordinator
Mar 3, 2015 at 7:57 PM
If you don't mind me asking, what prompted you to use Cygwin rather than the regular Python for Windows? We're looking into various scenarios that aren't adequately covered by the latter to see if something can be done to improve them.
Mar 5, 2015 at 3:25 AM
Edited Mar 5, 2015 at 3:38 AM
let me step in with more numerical use cases for cygwin: fortran compiler, sage and related cython debugging with gdb (MS is not interested). Here is a good reference from numpy folks, cygwin is mentioned more than MSVC:

https://github.com/numpy/numpy/wiki/Numerical-software-on-Windows
Coordinator
Mar 5, 2015 at 5:45 AM
We'd love to do Cython debugging (it's been on the wanted feature list since adding mixed mode). It just happens to be a very big feature to do right. But it's certainly going to happen at some point.

Regarding gdb (or lldb), we actually looked into the possibility of supporting that, seeing how VS 2015 actually added support for it for Android debugging. Unfortunately, gdb doesn't have the facilities that we need to implement true mixed mode - they didn't really make it with that kind of thing in mind, and there's only so much that can be done with hacks. So such debugging support would be limited to just Cython alone, and not work across the Python/Cython boundary.

Ideally, we'd love to get to the state where everything just works on Windows for scenarios like that (numeric computing, data science etc) without the need for Cygwin at all - so you'd only need it if you really want the Unix-like shell. Correct me if I'm wrong, but gfortran is also available in MinGW, isn't it? I'd imagine it's a PITA to set it all up, though, whereas in Cygwin you just install the requisite packages and everything can find everything else. So then the long-term fix for this would be making it just as easy to set this up with MinGW?
Mar 7, 2015 at 7:55 AM
pminaev,

Can you give the versions of python+numpy+windows that you managed to work?

Both numpy 1.7.2 (comes with cygwin) and 1.9.2 are not working with python2.7 in my box.

To answer your question, I generally feel cygwin is more dev friendly. I am trying to explore with numpy and scipy. Searching around the web, cygwin seems a simple choice. (except PSVT is not working well with it. :()

Thanks.
Coordinator
Mar 7, 2015 at 8:06 AM
I have installed all the packages via the Cygwin package manager. This is Python 2.7.8-1, and numpy 1.7.2-1.

The environment is configured as follows (in Tools -> Python Tools -> Environment Options):

Path: C:\cygwin64\bin\python2.7.exe
Windows Path: C:\cygwin64\bin\python2.7.exe
Library Path; C:\cygwin64\lib\python2.7
Architecture: x64
Language Version: 2.7

And then I'm running the following script:
import sys, os
print(sys.version)

os.environ['PATH'] = '/usr/local/bin:/usr/bin:/usr/lib/lapack:' + os.environ['PATH']
import numpy
and the import succeeds.
Coordinator
Mar 7, 2015 at 8:09 AM
FWIW, if you are just trying to explore numpy and scipy, on Windows, I think your best bet is a scientific distro that has them preinstalled, such as Anaconda or Canopy. Both are supported PTVS, and are largely zero friction.