py_proto_library messes up sibling subpackage imports with generated proto source

[mikex-oss]

🐞 bug report

Affected Rule

The issue is caused by the rule: py_proto_library

Is this a regression?

No, but did work in com_google_protobuf's py_proto_library rule (which we are trying to migrate from).

Description

Note: This seems somewhat related to #1551.

proto_library requires that the proto source be in the same package as the target. If we need to depend on an external repo's .proto file and create our own proto_library + py_proto_library targets, the workaround is to use a genrule.

Something about the generated source triggers the "_virtual_imports", even though #1551 (comment) seems to indicate this means we are definitively dealing with strip_import_prefix.

Unlike with strip_import_prefix, in this case, prepending the _virtual_imports/<proto_library_name> to the path causes other normal Python libraries along the path to fail. It seems the _virtual_imports should not be triggered for generated sources at all (this was the case for com_google_protobuf's version) or the imports needs to be updated to properly handle this case.

In the repro below, the generated protobuf Python library is at: bazel-bin/my_project/proto/_virtual_imports/foo_proto/my_project/proto/foo_pb2.py.

sys.path contains <execroot>/.../bin/my_project/proto/proto_fail.runfiles/<workspace_name>/my_project/proto/_virtual_imports/foo_proto before <execroot>/.../bin/my_project/proto/proto_fail.runfiles/<workspace_name>.

But _virtual_imports/foo_proto/ replicates the repo hierarchy to the proto target, i.e. it contains my_project/proto. This means that imports of other packages under my_project are picked from the former path entry and fail.

🔬 Minimal Reproduction

my_project/proto/BUILD

genrule(
    name = "generated_proto",
    srcs = ["@external_repo//path/to:foo.proto"],
    outs = ["foo.proto"],
    cmd = "cp $< $@",
)

proto_library(
    name = "foo_proto",
    srcs = ["foo.proto"],
)

py_proto_library(
    name = "foo_py_pb2",
    deps = [":foo_proto"],
)

py_binary(
    name = "proto_fail",
    srcs = [
        "proto_fail.py",
    ],
    deps = [
        "//my_project/api:bar,
        ":vertipro_py_pb2",
    ],
)

my_project/proto/proto_fail.py

from my_project.api import bar
from my_project.proto import foo_pb2

🔥 Exception or Error

$ bazel run //my_project/proto:proto_fail

bin/my_project/proto/proto_fail.runfiles/.../my_project/proto/proto_fail.py", line 1, in 
    from my_project.api import bar
ModuleNotFoundError: No module named 'my_project.api'

🌍 Your Environment

Operating System:

  
Debian Linux 6.5.6-1
  

Output of bazel version:

  
bazel 6.4.0
  

Rules_python version:

  
rules_python-0.27.1
  

Anything else relevant?