Support nested Source roots about pants

[ericzundel]

I want to provide a few details. The issue that I face that led to this report is a maven project with a second maven project nested beneath it.

/foo/pom.XML
/foo/BUILD
/foo/src/main/java
/foo/src/main/proto/
/foo/src/test/java

/foo/bar/pom.XML
/foo/bar/BUILD
/foo/bar/src/main/java
/foo/bar/src/main/proto
/foo/bar/src/test/java

The source roots are not really nested, but the project BUILD files where I defined java_library and java_protobuf_library are nested. The logic in sources.py assumes that given a target, you can find a single source root from it (before analyzing the source file list.) I took a quick stab at trying to infer the correct source root from the list of source files but hit a chicken and egg issue in some constructor calling SourceRoot.find()

[benjyw]

I think that patrick's currently in-flight refactor of BUILD file parsing
and the target hierarchy also re-does source roots. For example, it gets
rid of the shady logic we currently have where if it can't find an
appropriate source root when it parses a target it lazily evaluates BUILD
files by climbing up the directory ancestry until it finds a source_root()
that works.

After his refactor the source_roots won't be lazily discovered, they'll
have to be declared up front in BUILD.bootstrap or wherever.

So I expect that he'll be able to support this kind of thing pretty easily,
basically your source root will be whichever one is "closest" to you in the
ancestry. I think this is what you'd expect, and also supports your use
case?

On Thu, Apr 3, 2014 at 6:01 AM, Eric Ayers [email protected] wrote:

I want to provide a few details. The issue that I face that led to this
report is a maven project with a second maven project nested beneath it.

/foo/pom.XML
/foo/BUILD
/foo/src/main/java
/foo/src/main/proto/
/foo/src/test/java

/foo/bar/pom.XML
/foo/bar/BUILD
/foo/bar/src/main/java
/foo/bar/src/main/proto
/foo/bar/src/test/java

The source roots are not really nested, but the project BUILD files where
I defined java_library and java_protobuf_library are nested. The logic in
sources.py assumes that given a target, you can find a single source root
from it (before analyzing the source file list.) I took a quick stab at
trying to infer the correct source root from the list of source files but
hit a chicken and egg issue in some constructor calling SourceRoot.find()

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-39447696
.

[ericzundel]

I'll have to look at the code. The problem as I saw it was not from the lazy evaluation when the BUILD file is underneath two prospective source roots.

source_root('/foo/src/main/proto/src', java_protobuf_library)
source_root('/foo/bar/src/main/proto/src', java_protobuf_library)

If you had a target foo/BUILD that contained a java_protobuf_library(name="proto", ...), how would it choose between foo/src/main/proto/src and foo/bar/src/main/proto/src as the "source root" for the target?

This is currently important for figuring out the appropriate --proto-path arguments to protoc.

[benjyw]

Where are the sources of that java_protobuf_library?

On Thu, Apr 3, 2014 at 11:54 AM, Eric Ayers [email protected]:

I'll have to look at the code. The problem as I saw it was not from the
lazy evaluation when the BUILD file is underneath two prospective source
roots.

source_root('/foo/src/main/proto/src', java_protobuf_library)
source_root('/foo/bar/src/main/proto/src', java_protobuf_library)

If you had a target foo/BUILD that contained a
java_protobuf_library(name="proto", ...), how would it choose between
foo/src/main/proto/src and foo/bar/src/main/proto/src as the "source root"
for the target?

This is currently important for figuring out the appropriate --proto-path
arguments to protoc.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-39490411
.

[ericzundel]

The sources would be where ever you defined them to be in sources[], but in maven, they would be under src/main/proto beneath the pom.xml file:

foo/src/main/proto/com/squareup/foo/foo.proto contains

import "squareup/com/foo/bar/bar.proto"
...

foo/pom.xml # contains maven deps for the foo/ project
foo/BUILD:

java_protobuf_library(
name = 'proto',
sources = [
'./src/main/proto/com/squareup/foo/foo.proto',
],
dependencies = [
'foo/bar:proto',
],
)

/foo/bar/src/main/proto/com/squareup/foo/bar/bar.proto

foo/bar/pom.xml # contains maven deps for the foo/bar project
foo/bar/BUILD:

java_protobuf_library(
name = 'proto',
sources = [
'./src/main/proto/com/squareup/foo/bar/bar.proto',
],
)

[benjyw]

What I mean is, the answer to your question "how would it choose between
foo/src/main/proto/src and foo/bar/src/main/proto/src" should be that it'll
take whichever one is the closest ancestor to all of the sources.

Technically source_root should be a property of the payload, not of the
target.

On Thu, Apr 3, 2014 at 11:54 AM, Eric Ayers [email protected]:

I'll have to look at the code. The problem as I saw it was not from the
lazy evaluation when the BUILD file is underneath two prospective source
roots.

source_root('/foo/src/main/proto/src', java_protobuf_library)
source_root('/foo/bar/src/main/proto/src', java_protobuf_library)

If you had a target foo/BUILD that contained a
java_protobuf_library(name="proto", ...), how would it choose between
foo/src/main/proto/src and foo/bar/src/main/proto/src as the "source root"
for the target?

This is currently important for figuring out the appropriate --proto-path
arguments to protoc.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-39490411
.

[jsirois]

Assigning to @benjyw to close, but I think this can now be closed. Source roots are discovered unambiguously in overlapping cases now IIUC.

[benjyw]

Yes, source roots are discovered unambiguously, and the way they do so is configured via options. Bootstrap BUILD files are gone, as is the problematic logic in SourceRoots.find().

[gmalmquist]

Hooray!