Internal BoundsError in the compiler (in getfield_elim_pass!)

[iamed2]

Julia Version 1.6.0-DEV.706
Commit 92248d213e (2020-08-24 02:31 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake-avx512)

Internal error: encountered unexpected error in runtime:
BoundsError(a=Array{UInt64, (31,)}[0x0000000000000001, 0x0000000000000002, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000003, 0x0000000000000004, 0x0000000000000005, 0x0000000000000000, 0x0000000000000006, 0x0000000000000007, 0x0000000000000000, 0x0000000000000008, 0x0000000000000009, 0x0000000000000000], i=(32,))
jl_bounds_error_ints at /buildworker/worker/package_linux64/build/src/rtutils.c:183
getindex at ./array.jl:809 [inlined]
getindex at ./abstractarray.jl:1122 [inlined]
DFS at ./compiler/ssair/domtree.jl:197
SNCA at ./compiler/ssair/domtree.jl:269
construct_domtree at ./compiler/ssair/domtree.jl:121
getfield_elim_pass! at ./compiler/ssair/passes.jl:729
run_passes at ./compiler/ssair/driver.jl:131
optimize at ./compiler/optimize.jl:172
typeinf at ./compiler/typeinfer.jl:35
typeinf_edge at ./compiler/typeinfer.jl:536
abstract_call_method at ./compiler/abstractinterpretation.jl:465
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:129
abstract_call_known at ./compiler/abstractinterpretation.jl:1003
abstract_call at ./compiler/abstractinterpretation.jl:1026
abstract_call at ./compiler/abstractinterpretation.jl:1010
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1132
typeinf_local at ./compiler/abstractinterpretation.jl:1388
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1444
typeinf at ./compiler/typeinfer.jl:12
typeinf_edge at ./compiler/typeinfer.jl:536
abstract_call_method at ./compiler/abstractinterpretation.jl:465
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:129
abstract_call_known at ./compiler/abstractinterpretation.jl:1003
abstract_call at ./compiler/abstractinterpretation.jl:1026
abstract_call at ./compiler/abstractinterpretation.jl:1010
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1132
typeinf_local at ./compiler/abstractinterpretation.jl:1388
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1444
typeinf at ./compiler/typeinfer.jl:12
typeinf_edge at ./compiler/typeinfer.jl:536
abstract_call_method at ./compiler/abstractinterpretation.jl:465
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:129
abstract_call_known at ./compiler/abstractinterpretation.jl:1003
abstract_call at ./compiler/abstractinterpretation.jl:1026
abstract_call at ./compiler/abstractinterpretation.jl:1010
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1132
typeinf_local at ./compiler/abstractinterpretation.jl:1373
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1444
typeinf at ./compiler/typeinfer.jl:12
typeinf_edge at ./compiler/typeinfer.jl:536
abstract_call_method at ./compiler/abstractinterpretation.jl:465
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:129
abstract_call_known at ./compiler/abstractinterpretation.jl:1003
abstract_call at ./compiler/abstractinterpretation.jl:1026
abstract_call at ./compiler/abstractinterpretation.jl:1010
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1132
typeinf_local at ./compiler/abstractinterpretation.jl:1388
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1444
typeinf at ./compiler/typeinfer.jl:12
typeinf_edge at ./compiler/typeinfer.jl:536
abstract_call_method at ./compiler/abstractinterpretation.jl:465
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:129
abstract_call_known at ./compiler/abstractinterpretation.jl:1003
abstract_call at ./compiler/abstractinterpretation.jl:1026
abstract_call at ./compiler/abstractinterpretation.jl:1010
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1132
typeinf_local at ./compiler/abstractinterpretation.jl:1388
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1444
typeinf at ./compiler/typeinfer.jl:12
typeinf_edge at ./compiler/typeinfer.jl:536
abstract_call_method at ./compiler/abstractinterpretation.jl:465
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:129
abstract_call_known at ./compiler/abstractinterpretation.jl:1003
abstract_call at ./compiler/abstractinterpretation.jl:1026
abstract_call at ./compiler/abstractinterpretation.jl:1010
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1132
typeinf_local at ./compiler/abstractinterpretation.jl:1388
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1444
typeinf at ./compiler/typeinfer.jl:12
typeinf_edge at ./compiler/typeinfer.jl:536
abstract_call_method at ./compiler/abstractinterpretation.jl:465
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:129
abstract_call_known at ./compiler/abstractinterpretation.jl:1003
abstract_call at ./compiler/abstractinterpretation.jl:1026
abstract_call at ./compiler/abstractinterpretation.jl:1010
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1132
typeinf_local at ./compiler/abstractinterpretation.jl:1388
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1444
typeinf at ./compiler/typeinfer.jl:12
typeinf_edge at ./compiler/typeinfer.jl:536
abstract_call_method at ./compiler/abstractinterpretation.jl:465
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:129
abstract_call_known at ./compiler/abstractinterpretation.jl:1003
abstract_call at ./compiler/abstractinterpretation.jl:1026
abstract_call at ./compiler/abstractinterpretation.jl:1010
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1132
typeinf_local at ./compiler/abstractinterpretation.jl:1373
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1444
typeinf at ./compiler/typeinfer.jl:12
typeinf_edge at ./compiler/typeinfer.jl:536
abstract_call_method at ./compiler/abstractinterpretation.jl:465
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:129
abstract_call_known at ./compiler/abstractinterpretation.jl:1003
abstract_call at ./compiler/abstractinterpretation.jl:1026
abstract_call at ./compiler/abstractinterpretation.jl:1010
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1132
typeinf_local at ./compiler/abstractinterpretation.jl:1388
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1444
typeinf at ./compiler/typeinfer.jl:12
typeinf_ext at ./compiler/typeinfer.jl:623
typeinf_ext_toplevel at ./compiler/typeinfer.jl:656
typeinf_ext_toplevel at ./compiler/typeinfer.jl:652
jfptr_typeinf_ext_toplevel_9597 at /usr/local/julia/lib/julia/sys.so (unknown line)
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1753 [inlined]
jl_type_infer at /buildworker/worker/package_linux64/build/src/gf.c:300
jl_generate_fptr at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:301
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1888
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1839 [inlined]
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2143 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2334
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1753 [inlined]
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:117
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:206
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:436
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:491
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:491
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:491
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:491
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:659
macro expansion at /builds/invenia/GPForecasters.jl/test/miso_run.jl:211 [inlined]
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1114 [inlined]
macro expansion at /builds/invenia/GPForecasters.jl/test/miso_run.jl:191 [inlined]
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1114 [inlined]
top-level scope at /builds/invenia/GPForecasters.jl/test/miso_run.jl:23
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:838
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:788
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:881
eval at ./boot.jl:344 [inlined]
include_string at ./loading.jl:1104
_include at ./loading.jl:1158
include at ./client.jl:443
macro expansion at /builds/invenia/GPForecasters.jl/test/runtests.jl:27 [inlined]
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1189 [inlined]
macro expansion at /builds/invenia/GPForecasters.jl/test/runtests.jl:26 [inlined]
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1114 [inlined]
top-level scope at /builds/invenia/GPForecasters.jl/test/runtests.jl:16
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:832
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:788
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:881
eval at ./boot.jl:344 [inlined]
include_string at ./loading.jl:1104
_include at ./loading.jl:1158
include at ./client.jl:443
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1753 [inlined]
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:117
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:206
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:157 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:551
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:659
top-level scope at none:6
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:838
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:788
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:881
eval at ./boot.jl:344
exec_options at ./client.jl:260
_start at ./client.jl:484
jfptr__start_22849 at /usr/local/julia/lib/julia/sys.so (unknown line)
jl_apply at /buildworker/worker/package_linux64/build/ui/../src/julia.h:1753 [inlined]
true_main at /buildworker/worker/package_linux64/build/ui/repl.c:106
main at /buildworker/worker/package_linux64/build/ui/repl.c:227
__libc_start_main at /lib64/libc.so.6 (unknown line)
_start at /usr/local/julia/bin/julia (unknown line)

Originally posted by @iamed2 in #37154 (comment), but @vchuravy believes this is a different issue

[iamed2]

@iamed2 do you have a reproducer for that? It might be related to #37147

(also I suspect that it a different bug from the original report)

I don't have a reproducer I can share since this comes from one of our most complicated internal packages, but I can reproduce the issue in our CI and it'll run nightly using an updated nightly Julia build, so I'll be able to see if it's fixed.

[vchuravy]

Ok without a reproducer it's going to be hard to figure out.

First thing, can you comment out

julia/base/compiler/ssair/driver.jl

Line 128 in 304efd9

#@timeit "verify 2" verify_ir(ir)

?

and print out ir.cfg on

julia/base/compiler/ssair/driver.jl

Line 130 in 304efd9

#@Base.show ("before_sroa", ir)

[vchuravy]

Also can you revert the PR I mentioned locally or do a git bisect? I am also happy to take at private code (PM me on Slack)

[iamed2]

First thing, can you comment out

That's already commented out, do you want me to uncomment or leave it commented out?

[vchuravy]

Sorry uncomment it :)

[iamed2]

Uncommenting that timeit line and building causes an error during bootstrap:

    JULIA usr/lib/julia/corecompiler.ji
error during bootstrap:
LoadError(at "compiler/compiler.jl" line 3: LoadError(at "compiler/bootstrap.jl" line 8: MethodError(f=typeof(Core.Compiler.getindex)(), args=(Core.Compiler.NewNodeStream(stmts=Core.Compiler.InstructionStream(inst=Array{Any, (2,)}[
  Expr(:call, Core.getfield, SSAValue(1), 1),
  Expr(:call, Core.getfield, SSAValue(1), 2)], type=Array{Any, (2,)}[
  Any,
  Any], info=Array{Any, (2,)}[nothing, nothing], line=Array{Int32, (2,)}[1, 1], flag=Array{UInt8, (2,)}[0x00, 0x00]), info=Array{Core.Compiler.NewNodeInfo, (2,)}[Core.Compiler.NewNodeInfo(pos=2, attach_after=false), Core.Compiler.NewNodeInfo(pos=2, attach_after=false)]), 1), world=0x0000000000001016)))
jl_method_error_bare at /Users/ericdavies/repos/julia/src/gf.c:1740
jl_method_error at /Users/ericdavies/repos/julia/src/gf.c:1758
jl_lookup_generic_ at /Users/ericdavies/repos/julia/src/gf.c:2309
jl_apply_generic at /Users/ericdavies/repos/julia/src/gf.c:2330
unknown function (ip: 0x10a2155c4)
unknown function (ip: 0x10a20eb4a)
unknown function (ip: 0x10a208e47)
unknown function (ip: 0x10a0480e2)
unknown function (ip: 0x10a045504)
unknown function (ip: 0x109fb51ea)
unknown function (ip: 0x10a01a477)
unknown function (ip: 0x10a0161e3)
unknown function (ip: 0x10a008661)
unknown function (ip: 0x109fdab4a)
unknown function (ip: 0x109fd62ce)
unknown function (ip: 0x109fd5c69)
unknown function (ip: 0x109fc91ec)
unknown function (ip: 0x109fc1d5c)
unknown function (ip: 0x109fb85dc)
unknown function (ip: 0x109fb2a78)
unknown function (ip: 0x10a01a477)
unknown function (ip: 0x10a0161e3)
unknown function (ip: 0x10a008661)
unknown function (ip: 0x109fdab4a)
unknown function (ip: 0x109fd62ce)
unknown function (ip: 0x109fd5c69)
unknown function (ip: 0x109fc91ec)
unknown function (ip: 0x109fc1d5c)
unknown function (ip: 0x109fb85dc)
unknown function (ip: 0x109fb2a78)
unknown function (ip: 0x10a01a477)
unknown function (ip: 0x10a0161e3)
unknown function (ip: 0x10a008661)
unknown function (ip: 0x109fdab4a)
unknown function (ip: 0x109fd62ce)
unknown function (ip: 0x109fd5c69)
unknown function (ip: 0x109fc91ec)
unknown function (ip: 0x109fc0ea9)
unknown function (ip: 0x109fb85dc)
unknown function (ip: 0x109fb2a78)
unknown function (ip: 0x10a01a477)
unknown function (ip: 0x10a0161e3)
unknown function (ip: 0x10a008661)
unknown function (ip: 0x109fdab4a)
unknown function (ip: 0x109fd62ce)
unknown function (ip: 0x10a2d9d61)
unknown function (ip: 0x109fd8d00)
unknown function (ip: 0x109fd62ce)
unknown function (ip: 0x109fd5c69)
unknown function (ip: 0x109fc91ec)
unknown function (ip: 0x109fc1d5c)
unknown function (ip: 0x109fb85dc)
unknown function (ip: 0x109fb2a78)
unknown function (ip: 0x10a01a477)
unknown function (ip: 0x10a0161e3)
unknown function (ip: 0x10a008661)
unknown function (ip: 0x109fdab4a)
unknown function (ip: 0x109fd62ce)
unknown function (ip: 0x109fd5c69)
unknown function (ip: 0x109fc91ec)
unknown function (ip: 0x109fc0ea9)
unknown function (ip: 0x109fb85dc)
unknown function (ip: 0x109fb2a78)
unknown function (ip: 0x10a01a477)
unknown function (ip: 0x10a0161e3)
unknown function (ip: 0x10a008661)
unknown function (ip: 0x109fdab4a)
unknown function (ip: 0x109fd62ce)
unknown function (ip: 0x109fd5c69)
unknown function (ip: 0x109fc91ec)
unknown function (ip: 0x109fc1d5c)
unknown function (ip: 0x109fb85dc)
unknown function (ip: 0x109fb2a78)
unknown function (ip: 0x10a01a477)
unknown function (ip: 0x10a0161e3)
unknown function (ip: 0x10a008661)
unknown function (ip: 0x109fdab4a)
unknown function (ip: 0x109fd62ce)
unknown function (ip: 0x109fd5c69)
unknown function (ip: 0x109fc91ec)
unknown function (ip: 0x109fc0ea9)
unknown function (ip: 0x109fb85dc)
unknown function (ip: 0x109fb2a78)
unknown function (ip: 0x109f975a3)
unknown function (ip: 0x109f8b041)
unknown function (ip: 0x109f840ab)
jl_toplevel_eval_flex at /Users/ericdavies/repos/julia/src/toplevel.c:832
jl_parse_eval_all at /Users/ericdavies/repos/julia/src/toplevel.c:955
jl_load_ at /Users/ericdavies/repos/julia/src/toplevel.c:1002
unknown function (ip: 0x109f54032)
unknown function (ip: 0x109f53ef7)
jl_apply at /Users/ericdavies/repos/julia/src/./julia.h:1753 [inlined]
do_call at /Users/ericdavies/repos/julia/src/interpreter.c:117
eval_body at /Users/ericdavies/repos/julia/src/interpreter.c:0
jl_interpret_toplevel_thunk at /Users/ericdavies/repos/julia/src/interpreter.c:659
top-level scope at compiler/compiler.jl:126
jl_toplevel_eval_flex at /Users/ericdavies/repos/julia/src/toplevel.c:838
jl_eval_module_expr at /Users/ericdavies/repos/julia/src/toplevel.c:197
jl_toplevel_eval_flex at /Users/ericdavies/repos/julia/src/toplevel.c:664
jl_toplevel_eval at /Users/ericdavies/repos/julia/src/toplevel.c:847 [inlined]
jl_toplevel_eval_in at /Users/ericdavies/repos/julia/src/toplevel.c:881
unknown function (ip: 0x109f52ec2)
jl_apply at /Users/ericdavies/repos/julia/src/./julia.h:1753 [inlined]
do_call at /Users/ericdavies/repos/julia/src/interpreter.c:117
eval_body at /Users/ericdavies/repos/julia/src/interpreter.c:0
jl_interpret_toplevel_thunk at /Users/ericdavies/repos/julia/src/interpreter.c:659
top-level scope at compiler/compiler.jl:3
jl_toplevel_eval_flex at /Users/ericdavies/repos/julia/src/toplevel.c:838
jl_parse_eval_all at /Users/ericdavies/repos/julia/src/toplevel.c:955
jl_load_ at /Users/ericdavies/repos/julia/src/toplevel.c:1002
jl_load at /Users/ericdavies/repos/julia/src/toplevel.c:1015
exec_program at /Users/ericdavies/repos/julia/usr/bin/julia (unknown line)
true_main at /Users/ericdavies/repos/julia/usr/bin/julia (unknown line)
main at /Users/ericdavies/repos/julia/usr/bin/julia (unknown line)

make[1]: *** [/Users/ericdavies/repos/julia/usr/lib/julia/corecompiler.ji] Error 1
make: *** [julia-sysimg-ji] Error 2

[timholy]

Here's a reproducer that has worked 3/3 times I've tried it:

julia> @time using Makie
  5.524067 seconds (12.24 M allocations: 839.457 MiB, 4.93% gc time)

julia> @time display(plot(rand(5)))
Internal error: encountered unexpected error in runtime:
...

[maleadt]

Or ChainRules tests: https://github.com/JuliaCI/NanosoldierReports/blob/303a819e120eac6b87b3c74c18c024ae0359e457/pkgeval/by_hash/3573793_vs_c4acbf9/logs/ChainRules/1.6.0-DEV-4978cd5a4f.log

[KristofferC]

Or

Bukdu
ChainRulesTestUtils
GFlops
GitHubActions
SimpleMock
SolidStateDetectors
SparsityDetection
StochasticDelayDiffEq
StorageMirrorServer