Escape the path separator when prepending to PYTHONPATH - #207
Closed
DLuminary wants to merge 1 commit into
Closed
Conversation
PybindWrap.cmake and MatlabWrap.cmake build the environment for the wrapper
generator as
"PYTHONPATH=${GTWRAP_PACKAGE_DIR}${GTWRAP_PATH_SEPARATOR}$ENV{PYTHONPATH}"
with GTWRAP_PATH_SEPARATOR set to ":" on UNIX and ";" elsewhere. The UNIX
branch is fine. On the other branch an unescaped ";" is CMake's list
separator, so the quoted argument is split rather than being passed through
as a single PYTHONPATH assignment.
When PYTHONPATH is unset the string ends in a trailing ";", the empty second
element is dropped, and the command still receives one valid assignment --
which is why this has not been noticed. When PYTHONPATH is set, the split
lands in the middle of the command line and the build fails outright.
Reproduced with a minimal project using add_custom_command, forcing the
non-UNIX branch:
unescaped ";", PYTHONPATH unset -> ran ok
unescaped ";", PYTHONPATH set -> Error 127, build fails
escaped "\;", PYTHONPATH set -> ran ok, PYTHONPATH=/pkg/dir:/existing
escaped "\;", PYTHONPATH unset -> ran ok
escaped "\;", UNIX branch -> ran ok (unchanged)
So on Windows the wrapper currently builds only because PYTHONPATH is
typically unset; a conda environment or any other setup that defines it
would fail before the generator runs.
Note this is not tested on a real Windows host -- the non-UNIX branch was
forced on Linux to exercise the same CMake parsing. The UNIX path is
unaffected either way.
The same class of bug exists in gtsam's python/CMakeLists.txt, where the
four PYTHONPATH-setting targets join with "/" instead of a path-list
separator at all; that is fixed separately in borglab/gtsam.
Collaborator
|
Did you test under Windows? I tested with current CMake on Windows and this does not reproduce. |
Author
|
You're right — I didn't, and my test was invalid. Sorry. I forced the non-UNIX branch on Linux, which puts a ; into a Makefile recipe executed by /bin/sh. That's where the split happened, not in CMake. Reading the emitted recipe confirms it: CMake passes the whole string through as one argument, and the shell splits it at the ;. On Windows the recipe runs under cmd.exe, where ; isn't a command separator — so the branch is safe exactly where it actually runs, which is why you couldn't reproduce it. I also fixed it in borglab/gtsam#2739. Closing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PybindWrap.cmake and MatlabWrap.cmake build the environment for the wrapper generator as
with GTWRAP_PATH_SEPARATOR set to ":" on UNIX and ";" elsewhere. The UNIX branch is fine. On the other branch an unescaped ";" is CMake's list separator, so the quoted argument is split rather than being passed through as a single PYTHONPATH assignment.
When PYTHONPATH is unset the string ends in a trailing ";", the empty second element is dropped, and the command still receives one valid assignment -- which is why this has not been noticed. When PYTHONPATH is set, the split lands in the middle of the command line and the build fails outright.
Reproduced with a minimal project using add_custom_command, forcing the non-UNIX branch:
So on Windows the wrapper currently builds only because PYTHONPATH is typically unset; a conda environment or any other setup that defines it would fail before the generator runs.
Note this is not tested on a real Windows host -- the non-UNIX branch was forced on Linux to exercise the same CMake parsing. The UNIX path is unaffected either way.
The same class of bug exists in gtsam's python/CMakeLists.txt, where the four PYTHONPATH-setting targets join with "/" instead of a path-list separator at all; that is fixed separately in borglab/gtsam#2739.