Python 2/3 compat: gyb, gyb_sourcekit_support, gyb_syntax_support, li… #32109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

compnerd merged 4 commits into swiftlang:master from compnerd:behold-the-future

Jun 1, 2020

Member

compnerd commented May 30, 2020

…ne-directive

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

Member Author

compnerd commented May 30, 2020

CC: @gwynne @drodriguez @stephentyrone @drexin @shahmishal @edymtt

Member Author

compnerd commented May 30, 2020

@swift-ci please clean test

Member Author

compnerd commented May 30, 2020

This is a rebased change that should hopefully enable line-directive-tool and gyb to use python3. Note that this change on its own has no impact other than to make the code compatible with both versions, once this is merged, I can switch over the invocations of the tool to use python3 instead of python2.

Member Author

compnerd commented May 30, 2020

@drodriguez, @stephentyrone - you might remember this change from the original PR that @gwynne put up quite some time ago. I think that at this point we really do need to start migrating towards python 3. LLVM has decided that they will be dropping support for Python 2 at the end of the year, with the builds upstream now preferring python 3. Python 3 is easily available now (though there are some issues to be ironed out - @shahmishal and @JDevlieghere are probably better than I to explain the macOS specific issues). python.org has EOL'ed Python 2 at the beginning of this year. I'm hoping that we might be able to migrate the bulk of the day-to-day builds to python 3 in an incremental fashion.

Member Author

compnerd commented May 30, 2020

CC: @tachoknight

Contributor

swift-ci commented May 30, 2020

Build failed
Swift Test Linux Platform
Git Sha - d981de7f30ecdeb40b9c8125b4b65df4c0a68725

compnerd force-pushed the behold-the-future branch from d981de7 to 78ba971 Compare

May 31, 2020 00:10

Member Author

compnerd commented May 31, 2020

@swift-ci please test

Contributor

swift-ci commented May 31, 2020

Build failed
Swift Test OS X Platform
Git Sha - d981de7f30ecdeb40b9c8125b4b65df4c0a68725

Contributor

swift-ci commented May 31, 2020

Build failed
Swift Test OS X Platform
Git Sha - d981de7f30ecdeb40b9c8125b4b65df4c0a68725

Contributor

swift-ci commented May 31, 2020

Build failed
Swift Test Linux Platform
Git Sha - fb3a003882b54c6c2bd388576f1cbccb7d5e3ab4

compnerd force-pushed the behold-the-future branch from fb3a003 to eb8d1ad Compare

May 31, 2020 19:18

Member Author

compnerd commented May 31, 2020

@swift-ci please test

Member Author

compnerd commented May 31, 2020

@swift-ci please test Windows platform

Contributor

swift-ci commented May 31, 2020

Build failed
Swift Test OS X Platform
Git Sha - eb8d1ad0c9f07b8189b6bf52cf80ce7fb491fb6b

Contributor

swift-ci commented May 31, 2020

Build failed
Swift Test Linux Platform
Git Sha - eb8d1ad0c9f07b8189b6bf52cf80ce7fb491fb6b

gwynne and others added 4 commits

May 31, 2020 14:20


          Python 2/3 compat: gyb, gyb_sourcekit_support, gyb_syntax_support, li…

2001f07

…ne-directive


          gyb: make SyntaxSupport python3 compatible

e3c60e6

Adjust the python imports to be python2 and python3 compatible to enable
gyb to be switched over to python3.


          Python 2/3 compat: Swift stdlib

2127cbf


          stdlib: make the FloatingPoint GYB Python3 friendly

23f2d34

compnerd force-pushed the behold-the-future branch from eb8d1ad to 23f2d34 Compare

May 31, 2020 21:24

Member Author

compnerd commented May 31, 2020

@swift-ci please test

Contributor

swift-ci commented May 31, 2020

Build failed
Swift Test Linux Platform
Git Sha - 23f2d34

Member Author

compnerd commented May 31, 2020

@swift-ci please test Linux platform

Member Author

compnerd commented May 31, 2020

@swift-ci please test Windows platform

Contributor

swift-ci commented Jun 1, 2020

Build failed
Swift Test Linux Platform
Git Sha - 23f2d34

Member Author

compnerd commented Jun 1, 2020

@swift-ci please test Linux platform

Contributor

swift-ci commented Jun 1, 2020

Build failed
Swift Test Linux Platform
Git Sha - 23f2d34

Member Author

compnerd commented Jun 1, 2020

@swift-ci please test Linux platform

Contributor

swift-ci commented Jun 1, 2020

Build failed
Swift Test Linux Platform
Git Sha - 23f2d34

Member Author

compnerd commented Jun 1, 2020

@swift-ci please test Linux platform

Contributor

swift-ci commented Jun 1, 2020

Build failed
Swift Test Linux Platform
Git Sha - 23f2d34

Member Author

compnerd commented Jun 1, 2020

@swift-ci please test Linux platform

JDevlieghere approved these changes

View reviewed changes

compnerd merged commit 68a5343 into swiftlang:master

compnerd deleted the behold-the-future branch

June 1, 2020 21:18

Contributor

eeckstein commented Jun 3, 2020

@compnerd Looks like this broke the python lint test. I saw this on a PR test: https://ci.swift.org/job/swift-PR-osx/21352/console

Command Output (stdout):
--
./utils/gyb_syntax_support/__init__.py:4:1: I100 Import statements are in the wrong order. 'from . import Classification' should be before 'from .AvailabilityNodes import AVAILABILITY_NODES'
./utils/gyb_syntax_support/__init__.py:10:1: I100 Import statements are in the wrong order. 'from . import Token' should be before 'from .GenericNodes import GENERIC_NODES'
./utils/gyb_syntax_support/__init__.py:10:1: I202 Additional newline in a group of imports. 'from . import Token' is identified as Application Relative and 'from .GenericNodes import GENERIC_NODES' is identified as Application Relative.
./utils/gyb_syntax_support/__init__.py:12:1: I202 Additional newline in a group of imports. 'from .NodeSerializationCodes import SYNTAX_NODE_SERIALIZATION_CODES, get_serialization_code, verify_syntax_node_serialization_codes' is identified as Application Relative and 'from . import Token' is identified as Application Relative.
./utils/gyb_syntax_support/__init__.py:16:1: I202 Additional newline in a group of imports. 'from .PatternNodes import PATTERN_NODES' is identified as Application Relative and 'from .NodeSerializationCodes import SYNTAX_NODE_SERIALIZATION_CODES, get_serialization_code, verify_syntax_node_serialization_codes' is identified as Application Relative.
./utils/gyb_syntax_support/__init__.py:19:1: I202 Additional newline in a group of imports. 'from .Trivia import TRIVIAS' is identified as Application Relative and 'from .StmtNodes import STMT_NODES' is identified as Application Relative.

Though, I didn't see it on other builds.

Member Author

compnerd commented Jun 3, 2020

@eeckstein bleh, didn't seem to show up in the testing. I'll take a look, thanks for pointing this out!

Member Author

compnerd commented Jun 3, 2020

I wonder if its a python2 vs python3 issue

Contributor

tachoknight commented Jun 3, 2020

@compnerd i apologize in advance if it doesn't help, but i have a patch that I use when building Swift on Fedora for Python 3 that includes some gyb fixes. Hope there's something there that can help in this situation.

Member Author

compnerd commented Jun 3, 2020

@tachoknight unfortunately I had already replicated the issue and was working on fixing it. Thanks though!

#32157 should address the linter warnings

Member Author

compnerd commented Jun 3, 2020

@tachoknight - I think that it might be interesting/useful to see if you can drop any of that patch now. I suspect that it is possible. I think that with the other change that I have in flight we are pretty close to defaulting to python3 on the Swift build :-D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet