Merge branch 'master' into diskarrays

Conflicts:
	src/variable.jl

Author: tcarion

docs/src/attributes.md

@@ -23,7 +23,7 @@ typeof(attributes_as_dictionary)
 
 ## Possible type promotion in Julia
 
-There is a subtile problem with the following code:
+There is a subtle problem with the following code:
 
 ```julia
 ncv1 = defVar(ds,"v1", UInt8, ("longitude", "latitude", "time"), attrib = [
@@ -48,7 +48,7 @@ Julia effectively promotes the `_FillValue` to Float64 which leads to a `"NetCDF
 #   "_FillValue" => 255.0
 ```
 
-Note that the type of the second element of the `Pair`.
+Note the type of the second element of the `Pair`.
 
 Using a Julia `Dict` does not show this behaviour:
 

docs/src/dimensions.md

@@ -1,6 +1,6 @@
 # Dimensions
 
-In the NetCDF data model, dimension have names and a length (but possibly an unlimited length) and are defined for a NetCDF dataset (or group).
+In the NetCDF data model, dimensions have names and a length (but possibly an unlimited length) and are defined for a NetCDF dataset (or group).
 For a given `Variable` or `CFVariable`,the names of the corresponding dimensions are obtained with using [`dimnames`](@ref).
 
 ```@docs

docs/src/experimental.md

@@ -2,7 +2,7 @@
 
 ## Multi-file support
 
-Multiple files can also be aggregated over a given dimensions (or the record dimension). In this example, 3 sea surface temperature fields from the
+Multiple files can also be aggregated over a given dimension (or the record dimension). In this example, 3 sea surface temperature fields from the
 1992-01-01 to 1992-01-03 are aggregated using the OPeNDAP service from PODAAC.
 
 ```julia
@@ -51,7 +51,7 @@ name(ncbat)
 
 If there are multiple variables with the `standard_name` equal to `height`, an error is returned because it is ambiguous which variable should be accessed.
 
-All variables whose dimensions are also dimensions of `ncbat` are considered as related and can also be accessed by sub-setting `ncbar` with their variable names
+All variables whose dimensions are also dimensions of `ncbat` are considered as related and can also be accessed by sub-setting `ncbat` with their variable names
 of CF Standard name:
 
 ```jldoctest mylabel
@@ -67,7 +67,7 @@ The previous call to `ncbat[CF"longitude"]` would also worked if there are multi
 
 ## Views
 
-In Julia, a [view of an array](https://docs.julialang.org/en/v1/base/arrays/#Views-(SubArrays-and-other-view-types)) is a subset of an array but whose elements still point to the original parent array. If one modifies an element of a view, the corresponding element is the parent array is modified too:
+In Julia, a [view of an array](https://docs.julialang.org/en/v1/base/arrays/#Views-(SubArrays-and-other-view-types)) is a subset of an array but whose elements still point to the original parent array. If one modifies an element of a view, the corresponding element in the parent array is modified too:
 
 ```jldoctest example_view_julia
 A = zeros(4,4)
@@ -81,7 +81,7 @@ A[2,2]
 2.0
 ```
 
-Views do not use copy of the array. The parent array and the indices of the view and the obtained via the function [`parent`](https://docs.julialang.org/en/v1/base/arrays/#Base.parent) and [`parentindices`](https://docs.julialang.org/en/v1/base/arrays/#Base.parentindices).
+Views do not use copy of the array. The parent array and the indices of the view are obtained via the function [`parent`](https://docs.julialang.org/en/v1/base/arrays/#Base.parent) and [`parentindices`](https://docs.julialang.org/en/v1/base/arrays/#Base.parentindices).
 
 ```jldoctest example_view_julia
 parent(subset) == A
@@ -129,7 +129,7 @@ lon_subset[:] == [2, 3, 4]
 true
 ```
 
-A view of a NetCDF variable, also implement the function `parent` and `parentindices` with the same meaning as for julia `Array`s.
+A view of a NetCDF variable also implements the function `parent` and `parentindices` with the same meaning as for julia `Array`s.
 
 A whole dataset can also be sliced using a `view(ds, dim1=range1, dim2=range2...)`. For example:
 
@@ -149,8 +149,8 @@ Such sliced datasets can for example be saved into a new NetCDF file using `writ
 write("slice.nc",ds_subset)
 ```
 
-Any dimensions not mentioned in the `@view` call is not sliced.
-While `@view` produces a slices based on indices, the `NCDatasets.@select` macro produces a slice (of an NetCDF variable or dataset)
+Any dimension not mentioned in the `@view` call is not sliced.
+While `@view` produces a slice based on indices, the `NCDatasets.@select` macro produces a slice (of an NetCDF variable or dataset)
 based on the values of other related variables (typically coordinates).
 
 ## Data selection based on values

docs/src/index.md

@@ -75,7 +75,7 @@ Variables
 [...]
 ```
 
-This creates the central structure of NCDatasets.jl, `NCDataset`, which represents the contents of the netCDF file (without immediatelly loading everything in memory).
+This creates the central structure of NCDatasets.jl, `NCDataset`, which represents the contents of the netCDF file (without immediately loading everything in memory).
 
 The following displays the information just for the variable `varname`:
 
@@ -187,7 +187,7 @@ end
 ### Edit an existing netCDF file
 
 When you need to modify the variables or the attributes of a netCDF, you have
-to open it with the `"a"` option. Here of instance we add a global attribute *creator* to the
+to open it with the `"a"` option. Here, for instance, we add a global attribute *creator* to the
 file created in the previous step.
 
 ```julia
@@ -231,7 +231,7 @@ ncarea.attrib["units"] = "meter2";
 
 ### Get one or several variables by specifying the value of an attribute
 
-The variable name are not always standardized, for example the longitude we can
+The variable names are not always standardized. For example, for the longitude we can
 find: `lon`, `LON`, `longitude`, ...
 
 The solution implemented in the function `varbyattrib` consists in searching for the
@@ -242,7 +242,7 @@ nclon = varbyattrib(ds, standard_name = "longitude");
 ```
 will return the list of variables of the dataset `ds` that have "longitude"
 as standard name. To directly load the data of the first variable with the
-attribute `standard_name` equal to `"longitude"` one can the following:
+attribute `standard_name` equal to `"longitude"` one can do the following:
 
 ```julia
 data = varbyattrib(ds, standard_name = "longitude")[1][:]

docs/src/issues.md

@@ -146,7 +146,7 @@ Stacktrace:
 during initialization of module NetCDF_jll
 ```
 
-You will likely have similar issue with julia installed from other package manager (like Debian/Ubuntu apt, Homebrew...). The only supported solution is to install
+You will likely have similar issues with julia installed from other package managers (like Debian/Ubuntu apt, Homebrew...). The only supported solution is to install
 the [offical julia builds](https://julialang.org/downloads/).
 
 

docs/src/performance.md

@@ -1,18 +1,18 @@
 # [Performance tips](@id performance_tips)
 
-* Reading data from a file is not type-stable, because the type of the output of the read operation does depedent on the type defined in the NetCDF files and the value of various attribute (like `scale_factor`, `add_offset` and `units` for time conversion). All this information cannot be inferred from a static analysis of the source code. It is therefore recommended to use [type annotation](https://docs.julialang.org/en/v1/manual/types/index.html#Type-Declarations-1) if resulting type of a read operation in known:
+* Reading data from a file is not type-stable, because the type of the output of the read operation is dependent on the type defined in the NetCDF files and the value of various attribute (like `scale_factor`, `add_offset` and `units` for time conversion). All this information cannot be inferred from a static analysis of the source code. It is therefore recommended to use [type annotation](https://docs.julialang.org/en/v1/manual/types/index.html#Type-Declarations-1) if resulting type of a read operation in known:
 
 ```julia
 ds = NCDataset("file.nc")
 nctemp = ds["temp"]
 temp = nctemp[:,:] :: Array{Float64,2}
 # or
 # call_barrier_function(nctemp)
+
+Alternatively, one can also use so called "[function barriers](https://docs.julialang.org/en/v1/manual
 # call_barrier_function(temp)
 close(ds)
-```
-
-Alternatively, one can also use so called "[function barriers](https://docs.julialang.org/en/v1/manual/performance-tips/index.html#kernel-functions-1)" or the in-place `NCDatasets.load!` function (which is unexported, so it has to be prefixed with the module name):
+```/performance-tips/index.html#kernel-functions-1)" or the in-place `NCDatasets.load!` function (which is unexported, so it has to be prefixed with the module name):
 
 ```julia
 ds = NCDataset("file.nc")
@@ -36,7 +36,7 @@ ds = NCDataset("file.nc")
 close(ds)
 ```
 
-* Avoid when possible indexing with arrays and `CartesianIndex` as they also result in loading the data element-wise.
+* Avoid, when possible, indexing with arrays and `CartesianIndex` as they also result in loading the data element-wise.
 
 ```julia
 ds = NCDataset("dataset.nc");

docs/src/tutorials.md

@@ -98,7 +98,7 @@ variables:
 ```
 
 
-This is the typically error message which is returned when the credentials are not configured properly:
+This is the typical error message which is returned when the credentials are not configured properly:
 
 ```
 syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
@@ -157,9 +157,9 @@ This script produces the following plot:
 
 ## Access data from S3 object storage
 
-NASA EarthData is also available from AWS S3 object storage at the AWS region us-west-2. This example assumes that you have access to an AWS instance in this region (available thought e.g. [Pangeo Cloud](https://pangeo.io/cloud.html) after registration thanks to funding from the NASA's ACCESS/AWS Cloud Credits for Research program and the fine folks at [Pangeo](https://github.com/pangeo-data)). 
+NASA EarthData is also available from AWS S3 object storage at the AWS region us-west-2. This example assumes that you have access to an AWS instance in this region (available through e.g. [Pangeo Cloud](https://pangeo.io/cloud.html) after registration thanks to funding from the NASA's ACCESS/AWS Cloud Credits for Research program and the fine folks at [Pangeo](https://github.com/pangeo-data)).
 
-The S3 access token available from [https://archive.podaac.earthdata.nasa.gov/s3credentials](https://archive.podaac.earthdata.nasa.gov/s3credentials) are only valid for 1 hour. It is therefore preferable to get this token programmatically using your EarthData username and password. The function `earthdata_s3credentials` is a translation of ["Sample script to receive credentials"](https://web.archive.org/web/20220623090241/https://nasa.github.io/cumulus-distribution-api/#temporary-s3-credentials) to Julia:
+The S3 access tokens available from [https://archive.podaac.earthdata.nasa.gov/s3credentials](https://archive.podaac.earthdata.nasa.gov/s3credentials) are only valid for 1 hour. It is therefore preferable to get this token programmatically using your EarthData username and password. The function `earthdata_s3credentials` is a translation of ["Sample script to receive credentials"](https://web.archive.org/web/20220623090241/https://nasa.github.io/cumulus-distribution-api/#temporary-s3-credentials) to Julia:
 
 ```julia
 using AWS, Base64, JSON3, HTTP, NCDatasets, URIs

src/dataset.jl

@@ -192,7 +192,7 @@ function NCDataset(filename::AbstractString,
         elseif mode == "a"
             NC_WRITE
         elseif mode == "c"
-            ncmode  = NC_CLOBBER
+            NC_CLOBBER
         else
             throw(NetCDFError(-1, "Unsupported mode '$(mode)' for filename '$(filename)'"))
         end

src/variable.jl

@@ -92,6 +92,21 @@ function checkbuffer(len,data)
     end
 end
 
+@inline function unsafe_load!(ncvar::Variable, data, indices::Union{Integer, UnitRange, StepRange, Colon}...)
+    sizes = size(ncvar)
+    normalizedindices = normalizeindexes(sizes, indices)
+    ind = to_indices(ncvar,normalizedindices)
+
+    start,count,stride,jlshape = ncsub(ind)
+
+    @boundscheck begin
+        checkbounds(ncvar,indices...)
+        checkbuffer(prod(count),data)
+    end
+
+    nc_get_vars!(ncvar.ds.ncid,ncvar.varid,start,count,stride,data)
+end
+
 """
     NCDatasets.load!(ncvar::Variable, data, indices)
 
@@ -101,7 +116,8 @@ bounds checking can be elided by the compiler (which typically require
 type-stable code).
 
 ```julia
-ds = Dataset("file.nc")
+using NCDatasets
+ds = NCDataset("file.nc")
 ncv = ds["vgos"].var;
 # data must have the right shape and type
 data = zeros(eltype(ncv),size(ncv));
@@ -115,20 +131,18 @@ data = zeros(5); # must have the right shape and type
 load!(ds["temp"].var,data,:,1) # loads the 1st column
 ```
 
+!!! note
+
+    For a netCDF variable of type `NC_CHAR`, the element type of the `data`
+    array must be `UInt8` and cannot be the julia `Char` type, because the
+    julia `Char` type uses 4 bytes and the NetCDF `NC_CHAR` only 1 byte.
 """
 @inline function load!(ncvar::Variable{T,N}, data::AbstractArray{T}, indices::Union{Integer, UnitRange, StepRange, Colon}...) where {T,N}
-    sizes = size(ncvar)
-    normalizedindices = normalizeindexes(sizes, indices)
-    ind = to_indices(ncvar,normalizedindices)
-
-    start,count,stride,jlshape = ncsub(ind)
-
-    @boundscheck begin
-        checkbounds(ncvar,indices...)
-        checkbuffer(prod(count),data)
-    end
+    @inline unsafe_load!(ncvar, data, indices...)
+end
 
-    nc_get_vars!(ncvar.ds.ncid,ncvar.varid,start,count,stride,data)
+@inline function load!(ncvar::Variable{Char,N}, data::AbstractArray{UInt8}, indices::Union{Integer, UnitRange, StepRange, Colon}...) where N
+    @inline unsafe_load!(ncvar, data, indices...)
 end
 
 @inline function load!(ncvar::Variable{T,2}, data::AbstractArray{T}, i::Colon,j::UnitRange) where T
@@ -312,19 +326,19 @@ nomissing(a::AbstractArray) = a
 """
     a = nomissing(da,value)
 
-Retun the values of the array `da` of type `Array{Union{T,Missing},N}`
+Retun the values of the array `da` of type `AbstractArray{Union{T,Missing},N}`
 as a regular Julia array `a` by replacing all missing value by `value`
 (converted to type `T`).
 This function is identical to `coalesce.(da,T(value))` where T is the element
-tyoe of `da`.
+type of `da`.
 ## Example:
 
 ```julia-repl
 julia> nomissing([missing,1.,2.],NaN)
 # returns [NaN, 1.0, 2.0]
 ```
 """
-function nomissing(da::Array{Union{T,Missing},N},value) where {T,N}
+function nomissing(da::AbstractArray{Union{T,Missing},N},value) where {T,N}
     return replace(da, missing => T(value))
 end
 

test/test_https.jl

@@ -3,7 +3,7 @@ using Test
 
 sample_url = "https://rda.ucar.edu/thredds/dodsC/files/g/ds084.1/2018/20181231/gfs.0p25.2018123118.f003.grib2"
 
-@test_broken begin
+@test begin
     NCDataset(sample_url) do ds
         haskey(ds,"lon")
     end