Merge pull request #365 from ysb33r/rubygem-api

Improve GEM handling and resolving (#364, #366, #370)

Author: ysb33r

README.adoc

@@ -23,9 +23,10 @@ NOTE: JRuby/Gradle 2.0 relies on JRuby 9.x and later.
 
 This repository contains the:
 
-* `base` (in `jruby-gradle-base-plugin/`): dependency resolution, executing Ruby, etc.
-* `jar` (in `jruby-gradle-jar-plugin/`):  packaging JRuby-based `.jar` files
-* `war` (in `jruby-gradle-war-plugin/`): packaging JRuby-based `.war` files
+* `core` (in `core-plugin/`): Rubygems proxy.
+* `base` (in `base-plugin/`): dependency resolution, executing Ruby, etc.
+* `jar` (in `jar-plugin/`):  packaging JRuby-based `.jar` files
+* `war` (in `war-plugin/`): packaging JRuby-based `.war` files
 
 
 NOTE: More documentation can be found on link:http://jruby-gradle.org[jruby-gradle.org]

base-plugin/README.adoc

@@ -7,24 +7,16 @@ image::https://api.bintray.com/packages/jruby-gradle/plugins/jruby-gradle-plugin
 
 === Compatibility
 
-This plugin requires link:http://gradle.org[Gradle] 2.0 or better
-
+This plugin requires link:http://gradle.org[Gradle] 4.0 or better
 
 === Installing
 
 .build.gradle
 [source, groovy]
 ----
-buildscript {
-    repositories { jcenter() }
-
-    dependencies {
-        /* check jruby-gradle.org for the latest release */
-        classpath "com.github.jruby-gradle:jruby-gradle-plugin:[1.1.4,2.0)"
-    }
+plugins {
+  id 'com.github.jruby-gradle.base' version 'VERSION-OF-PLUGIN'
 }
-
-apply plugin: 'com.github.jruby-gradle.base'
 ----
 
 === Adding gems
@@ -44,15 +36,9 @@ dependencies {
 }
 ----
 
-IMPORTANT: Prereleased gems can only be a specific version due to limitations of mapping a rubygems repository to a maven repository.
+IMPORTANT: Prereleased gems should only be a specific version due to limitations of mapping a rubygems repository to an Ivy repository.
 
-NOTE: This functionality currently relies on a deployed version of the
-link:https://github.com/torquebox/rubygems-servlets[rubygems-servlets]
-software to help proxy a Rubygem repository into a Maven repository format.
-This ensures that Gradle can resolve gem-based dependencies. _Currently_ the
-plugin release on link:http://rubygems.lasagna.io/proxy/maven/releases[a
-Rubygems Maven proxy] operated by link:https://github.com/rtyler[R. Tyler
-Croy]
+NOTE: This functionality relies on an internal proxy which converts information from https://rubygems.org in to an Ivy repository service. this functionality is enabled by adding `ruby.gems()` to the `repositories` block. If you have a running Maven GEMs proxy you may use that too.
 
 == Tasks
 
@@ -89,16 +75,7 @@ Common methods for `JRubyExec` for executing a script
 * `standardOutput` - `OutputStream`.  Capture the output of the script.
 * `errorOutput` - `OutputStream`.  Capture the error output of the script.
 * `ignoreExitValue` - `Boolean`.  Ignore the JVm exit value. Exit values are only effective if the exit value of the Ruby script is correctly communicated back to the JVM.
-* `configuration` - `String`.  Configuration to copy gems from. (*)
-* `classpath` - `List`.  Additional Jars/Directories to place on classpath.
-* `jrubyVersion` - `String`.  JRuby version to use if not the same as ```project.jruby.execVersion```.
-* `gemWorkDir` - `File`. Provide a custom working directory for unpacking GEMs. By default each `JRubyExec` task
-  uses it's own work directory. Use this to set a common work directory for a number of tasks.
-
-(*) If `jrubyVersion` has not been set, `jrubyExec` will used as configuration.
-However, if `jrubyVersion` has been set, a configuration must also be provded
-to maintain dependency isolation, see
-link:http://jruby-gradle.org/errors/jrubyexec-version-conflict/[this page] for more details.
+* `jruby` - `JRubyPluginExtension`. Allows to override settigns from the global `jruby` extension.
 
 Additional `JRubyExec` methods for controlling the JVM instance
 
@@ -135,11 +112,8 @@ task needSomeRubyLove {
 
 The behaviour of `project.jrubyexec` is slightly different to that of `JRubyExec`.
 
-* The version of `jruby-complete` is strictly tied to the `jruby.execVersion`. Therefore trying to set `jrubyVersion`
-in the ```jrubyexec``` closure will cause a failure
-* GEMs and additional JARs are only taken from the `jrubyExec` configuration.
-* It is not possible to supply a `configuration` parameter to the `jrubyexec` closure.
-* GEMs will be installed to `jruby.gemInstallDir`. Existing gems will not be overwritten.
+* The version of JRuby is strictly tied to the global `jruby.jrubyVersion`.
+* GEMs and additional JARs are only taken from the `jruby.gemConfiguration` configuration.
 
 As with `JRubyExec`, `args`, `setArgs` and `main` are illegal within the `jrubyexec` closure.
 All other methods should work.
@@ -180,7 +154,11 @@ ext {
 
 link:/groovydoc/jruby-gradle-base-plugin/com/github/jrubygradle/JRubyPrepare.html[API docs]
 
-Unpacking occurs using the default `jruby` version as set by `jruby.execVersion`.
+Unpacking occurs using the default JRuby version as set by `jruby.jrubyVersion`. This can be overridden within the task itself if required.
+
+In most cases you do not have to do anything. The JRubyPrepare task will be created for you and associated with the appropriate configuration. If you customise configurations or use different configurations for different tasks, Gradle will useually create appropriate `JRubyPrepare` tasks for you and link the appropriate task dependencies.
+
+You can also create your own custom `JRubyPrepare` tasks if required.
 
 .build.gradle
 [source,gradle]
@@ -204,11 +182,9 @@ task unpackMyGems(type : JRubyPrepare) {
 
 === Using the built-in Gem proxy
 
-As of version 1.1.0, the JRuby/Gradle base plugin supports embedding a
-link:https://github.com/torquebox/rubygems-servlets[rubygems-servlets] process
-*inside* the Gradle process itself. This obviates the need for a third-party
+As of version 2.0, the JRuby/Gradle base plugin supports embedding a lightweight proxy inside* the Gradle process itself. This obviates the need for a third-party
 proxy to present the Maven-compatible dependency information that JRuby/Gradle
-relies on.
+relies on and it also supercedes the older link:https://github.com/torquebox/rubygems-servlets[rubygems-servlets] process
 
 .build.gradle
 [source,gradle]
@@ -220,43 +196,26 @@ jruby.defaultRepositories false
 
 repositories {
     jcenter()
-    rubygems('https://rubygems.org')
+    ruby.gems() // <1>
+    ruby.gems('https://your-own-server.org') // <2>
+    ruby.gems('https://another-server.org', 'acmegems') // <3>
+    ruby.torquebox() // <4>
 }
 
 dependencies {
-    gems 'rubygems:sinatra:1.4.5'
+    gems 'rubygems:sinatra:1.4.5' // <5>
+    gems 'acmegems:some-gem:1.2.3' // <6>
 }
 ----
+<1> Retrieve GEMs from https://rubygems.org
+<2> Retrieve GEMs from another server.
+<3> Retrieve GEMS from another server and associated a custom group name with that server.
+<4> Enabled the legacy Torquebox Maven proxy. Please note that this proxy server is unmaintained upstream and may lead to brittle builds in some cases.
+<5> Retrieve a GEM. `rubygems` is the default group for GEMs.
+<6> Retrieve a GEM using a custom group.
 
-=== Using a custom Gem repository
-
-By default the jruby plugin will use
-link:http://rubygems.lasanga.io[rubygems.lasanga.io] as its source of Ruby
-gems. This is a server operated by link:https://github.com/rtyler[R. Tyler
-Croy] and it presents a Maven repository of the data from
-link:https://rubygems.org[rubygems.org].
-
-If you **do not** wish to use this repository, you can run your own Maven proxy
-repository for either rubygems.org or your own gem repository by running the
-link:https://github.com/torquebox/rubygems-servlets[rubygems-servlets] server.
-
-You can then use that custom Gem repository with:
-
-.build.gradle
-[source,gradle]
-----
-jruby {
-    defaultRepositories false
-}
-
-repositories {
-    maven { url 'http://localhost:8989/releases' }
-}
-
-dependencies {
-    gems 'rubygems:my-custom-gem:1.0.1'
-}
-----
+NOTE: If the version of Gradle supports the `content` DSL keyword for repositories, only
+  requests dependencies with the specific groups will be sent to the GEM repositories.
 
 === Using the JRuby/Gradle without Gradle
 
@@ -277,3 +236,86 @@ end
 
 NOTE: The `.rb` file is assuming it's in the top level of the source tree, i.e.
 where `build.gradle` is located
+
+== Upgrading to 2.x from 1.7.x
+
+If you are already using the base proxy for work, upgrading will yield a number of surprises. Hopefully they will come as nice wow factors once you have changed your build file.
+
+=== Default repositories
+
+Repositories are no longer added by default. You should explictly declare which repositories you require. If you still need the old behaviour, you can obtain that by
+
+[source,groovy]
+----
+jruby {
+    defaultRepositories = true // <1>
+}
+----
+<1> Adds `ruby.gems()` and `jcenter()`.
+
+This functionality is deprecated and will be removed in a future version.
+
+=== jrubyExec configuration
+
+The `jrubyExec` configuration is no longer added as the GEM configuration handling has been cleaned up and much improved in 2.x. If you have a number of instances relying on `jrubyExec` you can do
+
+[source,groovy]
+----
+configurations {
+    jrubyExec
+}
+----
+
+and then enable this configuration either globally as your GEM configuration by doing
+
+[source,groovy]
+----
+jruby {
+    gemConfiguration 'jrubyExec'
+}
+----
+
+or on a task-specific basis
+
+[source,groovy]
+----
+task myRunner(type: JRubyExec) {
+    jruby {
+        gemConfiguration 'jrubyExec' // <1>
+    }
+}
+----
+<1> Enables configuration only for the given task.
+
+NOTE: If you set `gemConfiguration 'jrubyExec'` you will also notice that Gradle creates a `jrubyPrepareJrubyExec` tasks to unpack your GEMs in an isolated area.
+
+=== Setting GEM & JAR installation directories
+
+This behaviour was already deprecated in 1.7.x and has now been removed. Directories are now associated with specific configuration names. For the `gems` configuration this is in `${buildDir}/.gems`. For other configurations it will be a different directory. You can obtain the directory, via the `JRubyPrepare.getOutputDir()` method.
+
+=== Setting the JRuby version to use
+
+`jruby.defaultVersion` is deprecated. Do this rather via
+
+[source,groovy]
+----
+jruby {
+    jrubyVersion = '9.1.2.3'
+}
+----
+
+You can also override the JRuby version a per-task basis.
+
+[source,groovy]
+----
+task myRunner(type: JRubyExec) {
+    jruby {
+        jrubyVersion = '9.1.2.3' // <1>
+    }
+}
+----
+<1> Use a different version of JRuby only in a specific task, whilst using a globally configured version for everything else.
+
+=== Changing the JRubyExec configuration
+
+The `configuration` is deprecated. Use `jruby.gemConfiguration` on the task itself for a custom configuration.

base-plugin/build.gradle

@@ -29,8 +29,7 @@ ext {
 
 dependencies {
 
-    compile "org.eclipse.jetty:jetty-server:${jettyVersion}"
-    compile "org.eclipse.jetty:jetty-webapp:${jettyVersion}"
+    compile project(':jruby-gradle-core-plugin')
     runtime('de.saumya.mojo:rubygems:0.2.3@war') {
         // we just want the war file on the classloader for the application
         // to find it and use the war-file from filesystem
@@ -41,10 +40,12 @@ dependencies {
 
     testCompile(spockVersion) {
         exclude module: 'groovy-all'
+        exclude group: 'org.codehaus.groovy'
     }
 
     integrationTestCompile(spockVersion) {
         exclude module: 'groovy-all'
+        exclude group: 'org.codehaus.groovy'
     }
 
     // NOTE: This is used by JRubyPrepareGemsIntegrationSpec

base-plugin/src/gradleTest/jrubyExec/build.gradle

@@ -2,15 +2,20 @@ import com.github.jrubygradle.JRubyExec
 
 apply plugin : 'com.github.jruby-gradle.base'
 
+configurations {
+    jrubyExec
+}
+
 dependencies {
     jrubyExec ':credit_card_validator:1.1.0@gem'
 }
 
-jruby {
-    defaultRepositories = false
-}
 
 task requiresGems( type : JRubyExec ) {
+    jruby {
+        gemConfiguration 'jrubyExec'
+    }
+
     script 'scripts/requiresGem.rb'
 }
 

base-plugin/src/integTest/groovy/com/github/jrubygradle/JRubyExecExtensionIntegrationSpec.groovy

@@ -54,7 +54,7 @@ class JRubyExecExtensionIntegrationSpec extends IntegrationSpecification {
 
     void "Running a script that requires a jar"() {
         setup:
-        def leadingDir = 'jrubyExec/jars/org/bouncycastle/'
+        def leadingDir = '.gems/jars/org/bouncycastle/'
         def artifactPath = "${BCPROV_NAME}/${bcprovVer}/${BCPROV_NAME}-${bcprovVer}"
         def withPattern = ~/.*\["file:.+${leadingDir}${artifactPath}\.jar"\].*/
 
@@ -86,24 +86,6 @@ class JRubyExecExtensionIntegrationSpec extends IntegrationSpecification {
         result.output =~ /Not valid/
     }
 
-    void "Running a script that requires a gem, a separate jRuby, a separate configuration and a custom gemWorkDir"() {
-        setup:
-        final String customGemDir = 'customGemDir'
-        useScript(REQUIRES_GEM)
-        createJRubyExecProject withCreditCardValidator(), """
-            script '${REQUIRES_GEM}'
-            jrubyArgs '-T1'
-            gemWorkDir { new File(project.buildDir, '${customGemDir}' ) }
-        """
-
-        when:
-        BuildResult result = build()
-
-        then:
-        result.output =~ /Not valid/
-        new File(projectDir, "build/${customGemDir}").exists()
-    }
-
     void "Running a script that requires environment variables"() {
         // This tests that the passthrough invocation
         // happens for overloaded versions of environment
@@ -131,7 +113,7 @@ class JRubyExecExtensionIntegrationSpec extends IntegrationSpecification {
     }
 
     private BuildResult build() {
-        gradleRunner(DEFAULT_TASK_NAME, '-i').build()
+        gradleRunner(DEFAULT_TASK_NAME, '-i', '-s').build()
     }
 
     @SuppressWarnings('BuilderMethodWithSideEffects')
@@ -152,14 +134,16 @@ class JRubyExecExtensionIntegrationSpec extends IntegrationSpecification {
                     ${jrubyexecConfig}
                 }
             }
+
+            dependsOn jrubyPrepare
         }
         """
     }
 
     private String withJarToUse(String jarFormat) {
         """
             dependencies {
-                jrubyExec ${jarFormat}
+                gems ${jarFormat}
             }
         """
     }
@@ -171,4 +155,4 @@ class JRubyExecExtensionIntegrationSpec extends IntegrationSpecification {
     private String getBcprovVer() {
         testProperties.bcprovVersion
     }
-}
+}