Resolve more conflicts

Author: Saransh-cpp

docs/Project.toml

@@ -1,9 +1,14 @@
 [deps]
+BSON = "fbb218c0-5317-5bc6-957e-2ee96dd4b1f0"
+CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
 Functors = "d9f16b24-f501-4c13-a1f2-28368ffc5196"
 NNlib = "872c559c-99b0-510c-b3b7-b6c96a88d5cd"
+<<<<<<< HEAD
 MLUtils = "f1d291b0-491e-4a28-83b9-f70985020b54"
 CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
+=======
+>>>>>>> f437aabe (Add doctests for `recurrence.md`, `saving.md`, and `training.md`)
 
 [compat]
 Documenter = "0.26"

docs/make.jl

@@ -1,7 +1,7 @@
-using Documenter, Flux, NNlib, Functors, MLUtils, CUDA
+using Documenter, Flux, NNlib, Functors, MLUtils, CUDA, BSON
 
 DocMeta.setdocmeta!(Flux, :DocTestSetup, :(using Flux); recursive = true)
-makedocs(modules = [Flux, NNlib, Functors, MLUtils, CUDA],
+makedocs(modules = [Flux, NNlib, Functors, MLUtils, CUDA, BSON],
          doctest = VERSION == v"1.5",
          sitename = "Flux",
          pages = ["Home" => "index.md",

docs/src/models/recurrence.md

@@ -64,7 +64,9 @@ The `Recur` wrapper stores the state between runs in the `m.state` field.
 
 If we use the `RNN(2, 5)` constructor – as opposed to `RNNCell` – you'll see that it's simply a wrapped cell.
 
-```julia
+```jldoctest recurrence; setup = :(using Random; Random.seed!(0))
+julia> using Flux
+
 julia> RNN(2, 5)  # or equivalently RNN(2 => 5)
 Recur(
   RNNCell(2 => 5, tanh),                # 45 parameters
@@ -76,21 +78,28 @@ Equivalent to the `RNN` stateful constructor, `LSTM` and `GRU` are also availabl
 
 Using these tools, we can now build the model shown in the above diagram with: 
 
-```julia
-m = Chain(RNN(2 => 5), Dense(5 => 1))
+```jldoctest recurrence
+julia> m = Chain(RNN(2 => 5), Dense(5 => 1))
+Chain(
+  Recur(
+    RNNCell(2 => 5, tanh),              # 45 parameters
+  ),
+  Dense(5 => 1),                        # 6 parameters
+)         # Total: 6 trainable arrays, 51 parameters,
+          # plus 1 non-trainable, 5 parameters, summarysize 580 bytes.   
 ```
 In this example, each output has only one component.
 
 ## Working with sequences
 
 Using the previously defined `m` recurrent model, we can now apply it to a single step from our sequence:
 
-```julia
+```jldoctest recurrence
 julia> x = rand(Float32, 2);
 
 julia> m(x)
 1-element Vector{Float32}:
- 0.31759313
+ 0.45860028
 ```
 
 The `m(x)` operation would be represented by `x1 -> A -> y1` in our diagram.
@@ -102,14 +111,14 @@ iterating the model on a sequence of data.
 
 To do so, we'll need to structure the input data as a `Vector` of observations at each time step. This `Vector` will therefore be of `length = seq_length` and each of its elements will represent the input features for a given step. In our example, this translates into a `Vector` of length 3, where each element is a `Matrix` of size `(features, batch_size)`, or just a `Vector` of length `features` if dealing with a single observation.  
 
-```julia
+```jldoctest recurrence
 julia> x = [rand(Float32, 2) for i = 1:3];
 
 julia> [m(xi) for xi in x]
 3-element Vector{Vector{Float32}}:
- [-0.033448644]
- [0.5259023]
- [-0.11183384]
+ [0.36080405]
+ [-0.1391441]
+ [0.9310162]
 ```
 
 !!! warning "Use of map and broadcast"

docs/src/saving.md

@@ -6,7 +6,7 @@ session. The easiest way to do this is via
 
 Save a model:
 
-```julia
+```jldoctest saving; setup = :(using Random; Random.seed!(0))
 julia> using Flux
 
 julia> model = Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)
@@ -23,9 +23,7 @@ julia> @save "mymodel.bson" model
 
 Load it again:
 
-```julia
-julia> using Flux
-
+```jldoctest saving
 julia> using BSON: @load
 
 julia> @load "mymodel.bson" model
@@ -56,11 +54,13 @@ In some cases it may be useful to save only the model parameters themselves, and
 rebuild the model architecture in your code. You can use `params(model)` to get
 model parameters.
 
-```Julia
-julia> using Flux
-
+```jldoctest saving
 julia> model = Chain(Dense(10 => 5,relu),Dense(5 => 2),softmax)
-Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)
+Chain(
+  Dense(10 => 5, relu),                 # 55 parameters
+  Dense(5 => 2),                        # 12 parameters
+  NNlib.softmax,
+)                   # Total: 4 arrays, 67 parameters, 524 bytes.
 
 julia> weights = Flux.params(model);
 
@@ -72,10 +72,12 @@ julia> @save "mymodel.bson" weights
 You can easily load parameters back into a model with `Flux.loadparams!`.
 
 ```julia
-julia> using Flux
-
 julia> model = Chain(Dense(10 => 5,relu),Dense(5 => 2),softmax)
-Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)
+Chain(
+  Dense(10 => 5, relu),                 # 55 parameters
+  Dense(5 => 2),                        # 12 parameters
+  NNlib.softmax,
+)                   # Total: 4 arrays, 67 parameters, 524 bytes.
 
 julia> using BSON: @load
 
@@ -90,32 +92,39 @@ The new `model` we created will now be identical to the one we saved parameters
 
 In longer training runs it's a good idea to periodically save your model, so that you can resume if training is interrupted (for example, if there's a power cut). You can do this by saving the model in the [callback provided to `train!`](training/training.md).
 
-```julia
-using Flux: throttle
-using BSON: @save
+```jldoctest saving
+julia> using Flux: throttle
 
-m = Chain(Dense(10 => 5, relu), Dense(5 => 2), softmax)
+julia> using BSON: @save
+
+julia> m = Chain(Dense(10 => 5, relu), Dense(5 => 2), softmax)
+Chain(
+  Dense(10 => 5, relu),                 # 55 parameters
+  Dense(5 => 2),                        # 12 parameters
+  NNlib.softmax,
+)                   # Total: 4 arrays, 67 parameters, 524 bytes.
 
-evalcb = throttle(30) do
-  # Show loss
-  @save "model-checkpoint.bson" model
-end
+julia> evalcb = throttle(30) do
+         # Show loss
+         @save "model-checkpoint.bson" model
+       end
+(::Flux.var"#throttled#70"{Flux.var"#throttled#66#71"{Bool, Bool, var"#1#2", Int64}}) (generic function with 1 method)
 ```
 
 This will update the `"model-checkpoint.bson"` file every thirty seconds.
 
 You can get more advanced by saving a series of models throughout training, for example
 
 ```julia
-@save "model-$(now()).bson" model
+julia> @save "model-$(now()).bson" model
 ```
 
 will produce a series of models like `"model-2018-03-06T02:57:10.41.bson"`. You
 could also store the current test set loss, so that it's easy to (for example)
 revert to an older copy of the model if it starts to overfit.
 
 ```julia
-@save "model-$(now()).bson" model loss = testloss()
+julia> @save "model-$(now()).bson" model loss = testloss()
 ```
 
 Note that to resume a model's training, you might need to restore other stateful parts of your training loop. Possible examples are stateful optimizers (which usually utilize an `IdDict` to store their state), and the randomness used to partition the original data into the training and validation sets.
@@ -124,7 +133,7 @@ You can store the optimiser state alongside the model, to resume training
 exactly where you left off. BSON is smart enough to [cache values](https://github.com/JuliaIO/BSON.jl/blob/v0.3.4/src/write.jl#L71) and insert links when saving, but only if it knows everything to be saved up front. Thus models and optimizers must be saved together to have the latter work after restoring.
 
 ```julia
-opt = ADAM()
-@save "model-$(now()).bson" model opt
+julia> opt = ADAM()
+julia> @save "model-$(now()).bson" model opt
 ```
 

docs/src/training/training.md

@@ -116,9 +116,9 @@ A convenient way to run multiple epochs from the REPL is provided by `@epochs`.
 julia> using Flux: @epochs
 
 julia> @epochs 2 println("hello")
-INFO: Epoch 1
+[ Info: Epoch 1
 hello
-INFO: Epoch 2
+[ Info: Epoch 2
 hello
 
 julia> @epochs 2 Flux.train!(...)