Integration testing on master

[foxish]

Elaborating on #545 (comment)
We need to separate our integration testing out - so we can make it run any arbitrary branch - including master.

cc @apache-spark-on-k8s/contributors @ifilonenko @erikerlandson @ssuchter @kimoonkim

[erikerlandson]

are you thinking of factoring the IT into a separate repo?

[foxish]

I think that might be one way. If we just have the integration tests be built separately, as a project by itself, and then invoke it against master. It might not be the most expedient - open to other suggestions as well.

[erikerlandson]

One advantage of that would be that it would be convenient to make separate branches, having different IT subsets targeted to various upstreaming stages of functionality

[erikerlandson]

Would we then have to re-integrate them into the upstream so it has the correct IT?

[foxish]

It might be desirable for upstream also to have the same decoupled integration testing, similar to YARN/Mesos.

[erikerlandson]

+1, that makes it a two-fer

[erikerlandson]

maybe even a 3-fer, it keeps more code out of the main upstream PRs

[erikerlandson]

cc @felixcheung

[tnachen]

Having it outside of the main repo is easier and faster to iterate, which is why we did it also for Mesos..

[erikerlandson]

I added a tracking issue for jenkins-infra: https://github.com/ucbrise/jenkins-infra/issues/86

[felixcheung]

when you say decoupled testing, same as YARN/mesos, do you mean the test and source will be completely outside of the Apache Spark git repo?

(because as far as I can see, there is no open source YARN integration tests for Spark, for example)

[ssuchter]

But I’d think that not having OSS YARN integration tests is a bad thing. I think having some would have been good. In the current state, it’s hard for contributors that aren’t at a company that has their own private test suite to be confident of their changes, especially significant ones. I think we can have some for K8s, it’s a good thing. As mentioned in the thread, it doesn’t have to be in the same repo. The disadvantage of this, though, is that they are harder to keep in sync - if we change Spark in some way that enables/requires an integration test change, it becomes harder for every user of the integration tests. Personally, I think the advantages of a separate repo outweigh the disadvantage I listed above. I think we should go for a separate repo. Sean On November 30, 2017 at 10:29:25 PM, Felix Cheung (notifications@github.com) wrote: when you say decoupled testing, same as YARN/mesos, do you mean the test and source will be completely outside of the Apache Spark git repo? (because as far as I can see, there is no open source YARN integration tests for Spark, for example) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#568 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB2E24mGsg57hAvHIJkJR80AQlhGgm8Lks5s75zEgaJpZM4Qw9mJ> .

[ifilonenko]

I agree with the separate repo. However, this will require a group of separate group to maintain and monitor this repo as well.

[felixcheung]

If it's about fast iteration, I'd say go for it.
At some point though, I'd agree with @ssuchter OSS (or even, under ASF) and same branch/keep in-sync is very important.