Modernize install-julia.sh for CI and drop Julia v0.5

.travis.yml

@@ -43,6 +43,17 @@ matrix:
           - PYTHON_VERSION=py3
           - JULIA_VERSION=juliareleases
         os: osx
+  allow_failures:
+      - env: JULIA_VERSION=julianightlies
+      - env:
+          - JULIA_VERSION=julianightlies
+          - CROSS_VERSION=1
+      - env:
+          - PYTHON_VERSION=py2
+          - JULIA_VERSION=julianightlies
+      - env:
+          - PYTHON_VERSION=py3
+          - JULIA_VERSION=julianightlies
 notifications:
   email: false
 before_script:
@@ -51,8 +62,8 @@ before_script:
   - if [ "$TRAVIS_OS_NAME" = "linux" ]; then sudo apt-get update -qq -y; fi
   - if [ "$TRAVIS_OS_NAME" = "linux" ]; then sudo apt-get install libpcre3-dev -y; fi
   - if [ "$TRAVIS_OS_NAME" = "linux" ]; then sudo apt-get install python-numpy python3-numpy -y; fi
-  - if [ "$TRAVIS_OS_NAME" = "osx" -a "$PYTHON_VERSION" = "py3" ]; then brew list python3 &>/dev/null || brew install python3; fi
-  - if [ "$TRAVIS_OS_NAME" = "osx" -a "$PYTHON_VERSION" = "py2" ]; then brew list python &>/dev/null || brew install python; fi
+  - if [ "$TRAVIS_OS_NAME" = "osx" -a "$PYTHON_VERSION" = "py3" ]; then brew list python &>/dev/null || brew upgrade python; fi
+  - if [ "$TRAVIS_OS_NAME" = "osx" -a "$PYTHON_VERSION" = "py2" ]; then brew list python@2 &>/dev/null || brew install python@2; fi
 script:
   - julia -e 'Pkg.add("PyCall")'
   - julia -e 'Pkg.checkout("PyCall"); Pkg.build("PyCall")'
@@ -61,3 +72,4 @@ script:
   - python --version
   - if [ "$CROSS_VERSION" = "1" ]; then /usr/bin/python -m unittest discover; fi
   - python -m unittest discover
+

README.md

@@ -4,7 +4,7 @@ PyJulia
 [![Build Status](https://travis-ci.org/JuliaPy/pyjulia.svg?branch=master)](https://travis-ci.org/JuliaPy/pyjulia)
 [![Build status](https://ci.appveyor.com/api/projects/status/kjd0iex9gh0c3yqa?svg=true)](https://ci.appveyor.com/project/Keno/pyjulia)
 
-Experimenting with developing a better interface to [Julia language](https://julialang.org/) that works with [Python](https://www.python.org/) 2 & 3.
+Experimenting with developing a better interface to [Julia language](https://julialang.org/) that works with [Python](https://www.python.org/) 2 & 3 and Julia v0.6+.
 
 to run the tests, execute from the toplevel directory
 

appveyor.yml

@@ -2,62 +2,45 @@ environment:
   # for more python versions have a look at
   # https://github.com/ogrisel/python-appveyor-demo/blob/master/appveyor.yml
   matrix:
-  # - JULIAVERSION: "julialang/bin/winnt/x86/0.4/julia-0.4-latest-win32.exe"
-  #   PYTHONDIR: "C:\\Python27"
-  #   PYTHON: "C:\\Python27\\python.exe"
-  # - JULIAVERSION: "julialang/bin/winnt/x64/0.4/julia-0.4-latest-win64.exe"
-  #   PYTHONDIR: "C:\\Python27"
-  #   PYTHON: "C:\\Python27\\python.exe"
-  # - JULIAVERSION: "julianightlies/bin/winnt/x86/julia-latest-win32.exe"
-  #   PYTHONDIR: "C:\\Python27"
-  #   PYTHON: "C:\\Python27\\python.exe"
-
-  # 64 julia-0.5 Python-27
-  - JULIAVERSION: "julialang/bin/winnt/x64/0.5/julia-0.5-latest-win64.exe"
-    PYTHONDIR: "C:\\Python27-x64"
-
-  # 32 julia-0.5 Python-27
-  - JULIAVERSION: "julialang/bin/winnt/x86/0.5/julia-0.5-latest-win32.exe"
-    PYTHONDIR: "C:\\Python27"
-
-  # 64 julia-0.5 Python-35
-  - JULIAVERSION: "julialang/bin/winnt/x64/0.5/julia-0.5-latest-win64.exe"
-    PYTHONDIR: "C:\\Python35-x64"
-
   # 64 julia-0.6 Python-27
-  - JULIAVERSION: "julialang/bin/winnt/x64/0.6/julia-0.6-latest-win64.exe"
+  - JULIA_URL: "https://julialang-s3.julialang.org/bin/winnt/x64/0.6/julia-0.6-latest-win64.exe"
     PYTHONDIR: "C:\\Python27-x64"
 
   # 32 julia-0.6 Python-27
-  - JULIAVERSION: "julialang/bin/winnt/x86/0.6/julia-0.6-latest-win32.exe"
+  - JULIA_URL: "https://julialang-s3.julialang.org/bin/winnt/x86/0.6/julia-0.6-latest-win32.exe"
     PYTHONDIR: "C:\\Python27"
 
   # 64 julia-0.6 Python-35
-  - JULIAVERSION: "julialang/bin/winnt/x64/0.6/julia-0.6-latest-win64.exe"
+  - JULIA_URL: "https://julialang-s3.julialang.org/bin/winnt/x64/0.6/julia-0.6-latest-win64.exe"
     PYTHONDIR: "C:\\Python35-x64"
 
   # 32 julia-latest Python-27
-  - JULIAVERSION: "julianightlies/bin/winnt/x86/julia-latest-win32.exe"
+  - JULIA_URL: "https://julialangnightlies-s3.julialang.org/bin/winnt/x86/julia-latest-win32.exe"
     PYTHONDIR: "C:\\Python27"
 
   # 64 julia-latest Python-27
-  - JULIAVERSION: "julianightlies/bin/winnt/x64/julia-latest-win64.exe"
+  - JULIA_URL: "https://julialangnightlies-s3.julialang.org/bin/winnt/x64/julia-latest-win64.exe"
     PYTHONDIR: "C:\\Python27-x64"
 
   # 64 julia latest Python-35
-  - JULIAVERSION: "julianightlies/bin/winnt/x64/julia-latest-win64.exe"
+  - JULIA_URL: "https://julialangnightlies-s3.julialang.org/bin/winnt/x64/julia-latest-win64.exe"
     PYTHONDIR: "C:\\Python35-x64"
 
   # 64 julia latest Cross Version
-  - JULIAVERSION: "julianightlies/bin/winnt/x64/julia-latest-win64.exe"
+  - JULIA_URL: "https://julialangnightlies-s3.julialang.org/bin/winnt/x64/julia-latest-win64.exe"
     PYTHONDIR: "C:\\Python35-x64"
     CROSS_VERSION_PATH: "C:\\Python27-x64"
 
   # 32 julia latest Cross Version 
-  - JULIAVERSION: "julianightlies/bin/winnt/x86/julia-latest-win32.exe"
+  - JULIA_URL: "https://julialangnightlies-s3.julialang.org/bin/winnt/x86/julia-latest-win32.exe"
     PYTHONDIR: "C:\\Python35"
     CROSS_VERSION_PATH: "C:\\Python27"
 
+matrix:
+ allow_failures:
+  - JULIA_URL: "https://julialangnightlies-s3.julialang.org/bin/winnt/x86/julia-latest-win32.exe"
+  - JULIA_URL: "https://julialangnightlies-s3.julialang.org/bin/winnt/x64/julia-latest-win64.exe"
+
 branches:
   only:
     - master
@@ -74,18 +57,19 @@ install:
         https://ci.appveyor.com/api/projects/$env:APPVEYOR_ACCOUNT_NAME/$env:APPVEYOR_PROJECT_SLUG/history?recordsNumber=50).builds | `
         Where-Object pullRequestId -eq $env:APPVEYOR_PULL_REQUEST_NUMBER)[0].buildNumber) { `
         throw "There are newer queued builds for this pull request, failing early." }
-# Download most recent Julia Windows binary
+  # Download most recent Julia Windows binary
   - ps: (new-object net.webclient).DownloadFile(
-        $("http://s3.amazonaws.com/"+$env:JULIAVERSION),
+        $env:JULIA_URL,
         "C:\projects\julia-binary.exe")
-# Run installer silently, output to C:\projects\julia
+  # Run installer silently, output to C:\projects\julia
   - C:\projects\julia-binary.exe /S /D=C:\projects\julia
 
 build_script:
   - "SET PATH=%PYTHONDIR%;%PYTHONDIR%\\Scripts;%PATH%"
   - "\"C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\editbin.exe\" %PYTHONDIR%\\python.exe /STACK:8000000"
   # - C:\projects\julia\bin\julia -e "versioninfo(); Pkg.add(\"PyCall\"); Pkg.init(); Pkg.resolve()"
   # - C:\projects\julia\bin\julia -e "using PyCall; @assert isdefined(:PyCall); @assert typeof(PyCall) === Module"
+  - "SET PYTHON=%PYTHONDIR%\\python.exe"
   - C:\projects\julia\bin\julia -e "versioninfo(); Pkg.add(\"PyCall\"); Pkg.checkout(\"PyCall\"); Pkg.build(\"PyCall\")"
 
 test_script:

ci/install-julia.sh

@@ -2,6 +2,9 @@
 # install julia release: ./install-julia.sh juliareleases
 # install julia nightly: ./install-julia.sh julianightlies
 
+VERSION="0.6.2"
+SHORTVERSION="0.6"
+
 # stop on error
 set -e
 # default to juliareleases
@@ -13,13 +16,12 @@ fi
 
 case "$JULIAVERSION" in
   julianightlies)
-    STATUSURL="http://status.julialang.org/download"
+    BASEURL="https://julialangnightlies-s3.julialang.org/bin"
+    JULIANAME="julia-latest"
     ;;
   juliareleases)
-    STATUSURL="http://status.julialang.org/stable"
-    ;;
-  download/win32 | download/win64 | stable/win32 | stable/win64)
-    STATUSURL="http://status.julialang.org/$JULIAVERSION"
+    BASEURL="https://julialang-s3.julialang.org/bin"
+    JULIANAME="$SHORTVERSION/julia-$VERSION"
     ;;
   *)
     echo "Unrecognized JULIAVERSION=$JULIAVERSION, exiting"
@@ -29,22 +31,36 @@ esac
 
 case $(uname) in
   Linux)
-    if [ -e /usr/local/bin/julia ]; then
-      echo "/usr/local/bin/julia already exists, exiting"
-      exit 1
-    fi
     case $(uname -m) in
       x86_64)
-        curl -L "$STATUSURL/linux-x86_64" | tar -xz
+        ARCH="x64"
+        case "$JULIAVERSION" in
+          julianightlies)
+            SUFFIX="linux64"
+            ;;
+          juliareleases)
+            SUFFIX="linux-x86_64"
+            ;;
+        esac
         ;;
       i386 | i486 | i586 | i686)
-        curl -L "$STATUSURL/linux-i686" | tar -xz
+        ARCH="x86"
+        case "$JULIAVERSION" in
+          julianightlies)
+            SUFFIX="linux32"
+            ;;
+          juliareleases)
+            SUFFIX="linux-i686"
+            ;;
+        esac
         ;;
       *)
         echo "Do not have Julia binaries for this architecture, exiting"
         exit 1
         ;;
     esac
+    echo "$BASEURL/linux/$ARCH/$JULIANAME-$SUFFIX.tar.gz"
+    curl -L "$BASEURL/linux/$ARCH/$JULIANAME-$SUFFIX.tar.gz" | tar -xz
     sudo ln -s $PWD/julia-*/bin/julia /usr/local/bin/julia
     ;;
   Darwin)
@@ -58,7 +74,7 @@ case $(uname) in
       echo "~/julia already exists, exiting"
       exit 1
     fi
-    curl -Lo julia.dmg "$STATUSURL/osx10.7+"
+    curl -Lo julia.dmg "$BASEURL/mac/x64/$JULIANAME-mac64.dmg"
     hdiutil mount -mountpoint /Volumes/Julia julia.dmg
     cp -Ra /Volumes/Julia/*.app/Contents/Resources/julia ~
     ln -s ~/julia/bin/julia /usr/local/bin/julia
@@ -68,4 +84,4 @@ case $(uname) in
     echo "Do not have Julia binaries for this platform, exiting"
     exit 1
     ;;
-esac
+esac

julia/core.py

@@ -254,7 +254,7 @@ def __init__(self, init_julia=True, jl_runtime_path=None, jl_init_path=None,
             juliainfo = subprocess.check_output(
                 [runtime, "-e",
                  """
-                 println(JULIA_HOME)
+                 println(VERSION < v"0.7.0-DEV.3073" ? JULIA_HOME : Base.Sys.BINDIR)
                  println(Libdl.dlpath(string("lib", splitext(Base.julia_exename())[1])))
                  println(unsafe_string(Base.JLOptions().image_file))
                  PyCall_depsfile = Pkg.dir("PyCall","deps","deps.jl")
@@ -291,7 +291,8 @@ def __init__(self, init_julia=True, jl_runtime_path=None, jl_init_path=None,
             if use_separate_cache:
                 PYCALL_JULIA_HOME = os.path.join(
                     os.path.dirname(os.path.realpath(__file__)),"fake-julia").replace("\\","\\\\")
-                os.environ["JULIA_HOME"] = PYCALL_JULIA_HOME
+                os.environ["JULIA_HOME"] = PYCALL_JULIA_HOME  # TODO: this line can be removed when dropping Julia v0.6
+                os.environ["JULIA_BINDIR"] = PYCALL_JULIA_HOME
                 jl_init_path = PYCALL_JULIA_HOME.encode("utf-8")
 
             if not hasattr(self.api, "jl_init_with_image"):

julia/fake-julia/README

@@ -23,7 +23,7 @@ Since "normal" precompilation launches a new `julia` process, this process would
 (`libpython`) version of the PyCall cache file.    So, we have to force precompilation to launch
 a `python` process, not a `julia` process, so that PyCall is compiled correctly for running inside `python`.
 
-That is what `fake-julia` does.   By changing the `JULIA_HOME` environment variable, we trick Julia
+That is what `fake-julia` does.   By changing the `JULIA_HOME` (v0.6) or `JULIA_BINDIR` (v0.7+) environment variable, we trick Julia
 into launching `fake-julia/julia` instead of the "real" `julia` process during precompilation.  `fake-julia/julia`
 is actually a Python script, but it links `libjulia` and uses `libjulia` to process the command-line arguments,
 so it mimics the behavior of the `julia` process.  Since `fake-julia/julia` is running from within the `python`

julia/fake-julia/julia-win.c

@@ -9,7 +9,8 @@ int wmain(int argc, wchar_t *argv[], wchar_t *envp) {
 #else
 	SetEnvironmentVariableW(L"PYCALL_JULIA_FLAVOR",L"julia");
 #endif
-	SetEnvironmentVariableW(L"JULIA_HOME",_wgetenv(L"PYCALL_JULIA_HOME"));
+	SetEnvironmentVariableW(L"JULIA_HOME",_wgetenv(L"PYCALL_JULIA_HOME"));  // TODO: this can be removed when dropping Julia v0.6
+	SetEnvironmentVariableW(L"JULIA_BINDIR",_wgetenv(L"PYCALL_JULIA_HOME"));
 	wchar_t *python_process = _wgetenv(L"PYCALL_PYTHON_EXE");
 	if (python_process == NULL)
 		python_process = L"python";

julia/fake-julia/julia.py

@@ -11,7 +11,8 @@
     sh_ext = ".so"
 libjulia_path = os.environ["PYCALL_LIBJULIA_PATH"] + "/lib" + os.environ["PYCALL_JULIA_FLAVOR"] + sh_ext
 libjulia = ctypes.PyDLL(libjulia_path, ctypes.RTLD_GLOBAL)
-os.environ["JULIA_HOME"] = os.environ["PYCALL_JULIA_HOME"]
+os.environ["JULIA_HOME"] = os.environ["PYCALL_JULIA_HOME"]  # TODO: this can be removed when dropping Julia v0.6
+os.environ["JULIA_BINDIR"] = os.environ["PYCALL_JULIA_HOME"]
 
 if not hasattr(libjulia, "jl_init_with_image"):
     if hasattr(libjulia, "jl_init_with_image__threading"):