Skip to content

Commit 6fcd8cc

Browse files
author
gadamc
committed
Updates READMEs.
Improves warning message about sparklyr installation. Clarifies, for both packages, that this package has been tested to work with IBM Spark services and should work with other Spark services that utilize the swift2d/stocator library.
1 parent 51db705 commit 6fcd8cc

File tree

2 files changed

+28
-14
lines changed

2 files changed

+28
-14
lines changed

r/sparklyr/README.md

Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,30 @@
22

33
The package sets Spark Hadoop configurations for connecting to
44
IBM Bluemix Object Storage and Softlayer Account Object Storage instances
5-
with the swift protocol. This packages uses the new [swift2d/stocator](https://github.com/SparkTC/stocator) protocol, availble
6-
on the latest IBM Spark Service instances (and through IBM Data Science Experience).
7-
8-
Note, this package configures a SparkContext instantiated by SparkR and is appropriate for use
9-
with IBM R Notebooks only. It does not support SparkContexts returned from sparklyr::spark_connect, and
10-
can therefore not be used in IBM DSX RStudio sessions. Support for RStudio will come in the near future.
5+
with the swift protocol. This packages uses the new
6+
[swift2d/stocator](https://github.com/SparkTC/stocator) protocol, availble
7+
on the latest IBM Spark Service instances and through IBM Data Science Experience (DSX).
118

9+
This package expects a SparkContext instantiated by sparklyr. It has been tested
10+
to work with IBM RStudio from DSX, though it should work with other Spark
11+
installations that utilize the [swift2d/stocator](https://github.com/SparkTC/stocator).
1212

1313
## Installation
14+
1415
library(devtools)
15-
devtools::install_url("https://github.com/ibm-cds-labs/ibmos2spark/archive/adding_r_scala_platform.zip", subdir= "r/sparklyr/",dependencies = FALSE)
16-
17-
In Data Science Experience please be sure to include the "dependencies = FALSE" flag in your installation. If you forget to do this and DSX's special flavor of sparklyr gets overwritten, just go into your local R repo store ("/home/rstudio/R/x86_64-redhat-linux-gnu-library/RVERSION/") where RVERSION is the newest install of R (currently 3.3) and delete the sparklyr folder. After deleting choose File->Quit Session to refresh your R kernel. These steps will refresh your sparklyr package to the special Data Science Experience version.
16+
devtools::install_url("https://github.com/ibm-cds-labs/ibmos2spark/archive/<version>.zip", subdir= "r/sparklyr/",dependencies = FALSE)
17+
18+
where `version` should be a tagged release, such as `0.0.7`. (If you're daring, you can use `master`.)
19+
20+
###### WARNING
21+
22+
In IBM Data Science Experience, please be sure to include the `dependencies = FALSE` flag when
23+
calling `devtools::install_url`. If you forget to do this, you will most likely overwrite DSX's
24+
special flavor of sparklyr, which will break your connection to IBM Spark Services. To repair this,
25+
go into your local R repo store ("/home/rstudio/R/x86_64-redhat-linux-gnu-library/RVERSION/")
26+
where RVERSION is the newest install of R (currently 3.3) and delete the `sparklyr` folder.
27+
After deleting, choose File->Quit Session to refresh your R kernel. These steps will refresh your
28+
sparklyr package to the special DSX version.
1829

1930
## Usage
2031

r/sparkr/README.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,19 @@
33
The package sets Spark Hadoop configurations for connecting to
44
IBM Bluemix Object Storage and Softlayer Account Object Storage instances
55
with the swift protocol. This packages uses the new [swift2d/stocator](https://github.com/SparkTC/stocator) protocol, availble
6-
on the latest IBM Spark Service instances (and through IBM Data Science Experience).
6+
on the latest IBM Spark Service instances, and through IBM Data Science Experience.
77

8-
Note, this package configures a SparkContext instantiated by SparkR and is appropriate for use
9-
with IBM R Notebooks only. It does not support SparkContexts returned from sparklyr::spark_connect, and
10-
can therefore not be used in IBM DSX RStudio sessions. Support for RStudio will come in the near future.
8+
This package expects a SparkContext instantiated by SparkR. It has been tested to work with
9+
IBM Spark service in R notebooks on IBM DSX, though it should work with other Spark installations
10+
that utilize the [swift2d/stocator](https://github.com/SparkTC/stocator) protocol.
1111

1212

1313
## Installation
14+
1415
library(devtools)
15-
devtools::install_url("https://github.com/ibm-cds-labs/ibmos2spark/archive/adding_r_scala_platform.zip", subdir= "r/sparkr/")
16+
devtools::install_url("https://github.com/ibm-cds-labs/ibmos2spark/archive/<version).zip", subdir= "r/sparkr/")
17+
18+
where `version` should be a tagged release, such as `0.0.7`. (If you're daring, you can use `master`.)
1619

1720
## Usage
1821

0 commit comments

Comments
 (0)