diff --git a/docs/src/notebooks/MLJTutorial/01_data_representation/notebook.md b/docs/src/notebooks/MLJTutorial/01_data_representation/notebook.md
index ca841c7..63dd2b0 100644
--- a/docs/src/notebooks/MLJTutorial/01_data_representation/notebook.md
+++ b/docs/src/notebooks/MLJTutorial/01_data_representation/notebook.md
@@ -290,7 +290,7 @@ csv_file = Downloads.download(url)
````
````
-"/tmp/jl_nAeUTc/horse.csv"
+"/tmp/jl_ruz0kP/horse.csv"
````
Entering these lines of code downloads the data to a temporary file at the location
@@ -531,8 +531,8 @@ A = rand(2, 3)
````
2×3 Matrix{Float64}:
- 0.84235 0.324562 0.731728
- 0.233998 0.746378 0.485095
+ 0.113155 0.33251 0.397839
+ 0.171581 0.90293 0.436722
````
````@julia
@@ -550,8 +550,8 @@ Asparse = sparse(A)
````
2×3 SparseArrays.SparseMatrixCSC{Float64, Int64} with 6 stored entries:
- 0.84235 0.324562 0.731728
- 0.233998 0.746378 0.485095
+ 0.113155 0.33251 0.397839
+ 0.171581 0.90293 0.436722
````
````@julia
diff --git a/docs/src/notebooks/MLJTutorial/02_models/learning_curve.png b/docs/src/notebooks/MLJTutorial/02_models/learning_curve.png
index 85b533a..7234cf4 100755
Binary files a/docs/src/notebooks/MLJTutorial/02_models/learning_curve.png and b/docs/src/notebooks/MLJTutorial/02_models/learning_curve.png differ
diff --git a/docs/src/notebooks/MLJTutorial/02_models/notebook.md b/docs/src/notebooks/MLJTutorial/02_models/notebook.md
index 13a1648..c2bde2b 100644
--- a/docs/src/notebooks/MLJTutorial/02_models/notebook.md
+++ b/docs/src/notebooks/MLJTutorial/02_models/notebook.md
@@ -625,8 +625,8 @@ mach = machine(model, X, y)
untrained Machine; caches model-specific representations of data
model: NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …)
args:
- 1: Source @763 ⏎ ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}}
- 2: Source @104 ⏎ AbstractVector{ScientificTypesBase.Multiclass{3}}
+ 1: Source @909 ⏎ ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}}
+ 2: Source @399 ⏎ AbstractVector{ScientificTypesBase.Multiclass{3}}
````
@@ -652,18 +652,18 @@ fit!(mach, rows=train, verbosity=2);
````
[ Info: Training machine(NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …), …).
[ Info: MLJFlux: converting input data to Float32
-[ Info: Loss is 1.564
-[ Info: Loss is 1.405
-[ Info: Loss is 1.245
-[ Info: Loss is 1.125
-[ Info: Loss is 1.073
-[ Info: Loss is 1.018
-[ Info: Loss is 1.019
-[ Info: Loss is 0.9751
-[ Info: Loss is 1.015
-[ Info: Loss is 0.9778
-[ Info: Loss is 0.9424
-[ Info: Loss is 0.9263
+[ Info: Loss is 2.023
+[ Info: Loss is 1.592
+[ Info: Loss is 1.491
+[ Info: Loss is 1.205
+[ Info: Loss is 1.227
+[ Info: Loss is 1.177
+[ Info: Loss is 1.19
+[ Info: Loss is 1.128
+[ Info: Loss is 1.09
+[ Info: Loss is 1.058
+[ Info: Loss is 1.059
+[ Info: Loss is 1.056
````
@@ -676,9 +676,9 @@ yhat[1:3]
````
3-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, String, UInt32, Float32}:
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.326, Iris-versicolor=>0.331, Iris-virginica=>0.343)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.533, Iris-versicolor=>0.291, Iris-virginica=>0.177)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.524, Iris-versicolor=>0.293, Iris-virginica=>0.183)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.38, Iris-versicolor=>0.293, Iris-virginica=>0.327)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.449, Iris-versicolor=>0.271, Iris-virginica=>0.281)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.454, Iris-versicolor=>0.27, Iris-virginica=>0.276)
````
We'll have more to say on the form of this prediction shortly.
@@ -701,7 +701,7 @@ report(mach)
````
````
-(training_losses = Float32[1.2405895, 1.5638204, 1.4046175, 1.2446647, 1.125359, 1.0727392, 1.0180935, 1.0194263, 0.9750622, 1.0149823, 0.9778134, 0.94241244, 0.9262781],)
+(training_losses = Float32[1.7850709, 2.023422, 1.5918993, 1.4905419, 1.2054352, 1.2269526, 1.1768, 1.1895036, 1.128155, 1.0898077, 1.0582713, 1.0594574, 1.056259],)
````
You save a machine like this:
@@ -720,9 +720,9 @@ yhat[1:3]
````
3-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, String, UInt32, Float32}:
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.314, Iris-versicolor=>0.33, Iris-virginica=>0.356)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.531, Iris-versicolor=>0.291, Iris-virginica=>0.178)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.322, Iris-versicolor=>0.332, Iris-virginica=>0.346)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.379, Iris-versicolor=>0.304, Iris-virginica=>0.317)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.454, Iris-versicolor=>0.269, Iris-virginica=>0.277)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.377, Iris-versicolor=>0.292, Iris-virginica=>0.331)
````
Machines remember the last set of hyperparameters used during fit,
@@ -737,10 +737,10 @@ fit!(mach, rows=train, verbosity=2);
````
[ Info: Updating machine(NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …), …) (with warm restart if possible).
-[ Info: Loss is 0.926
-[ Info: Loss is 0.9323
-[ Info: Loss is 0.8731
-[ Info: Loss is 0.8909
+[ Info: Loss is 1.067
+[ Info: Loss is 1.038
+[ Info: Loss is 1.034
+[ Info: Loss is 1.038
````
@@ -771,10 +771,10 @@ fit!(mach, rows=train, verbosity=2);
````
[ Info: Updating machine(NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …), …) (with warm restart if possible).
-[ Info: Loss is 0.8501
-[ Info: Loss is 0.8156
-[ Info: Loss is 0.7756
-[ Info: Loss is 0.7181
+[ Info: Loss is 0.9739
+[ Info: Loss is 0.9423
+[ Info: Loss is 0.9133
+[ Info: Loss is 0.8122
````
@@ -789,26 +789,26 @@ fit!(mach, rows=train, verbosity=2);
````
[ Info: Updating machine(NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …), …) (with warm restart if possible).
[ Info: MLJFlux: converting input data to Float32
-[ Info: Loss is 1.152
-[ Info: Loss is 0.9842
-[ Info: Loss is 0.948
-[ Info: Loss is 0.8713
-[ Info: Loss is 0.8544
-[ Info: Loss is 0.8761
-[ Info: Loss is 0.7668
-[ Info: Loss is 0.7384
-[ Info: Loss is 0.6662
-[ Info: Loss is 0.6725
-[ Info: Loss is 0.7011
-[ Info: Loss is 0.6599
-[ Info: Loss is 0.6516
-[ Info: Loss is 0.6764
-[ Info: Loss is 0.6492
-[ Info: Loss is 0.6193
-[ Info: Loss is 0.6555
-[ Info: Loss is 0.6376
-[ Info: Loss is 0.6776
-[ Info: Loss is 0.7155
+[ Info: Loss is 1.285
+[ Info: Loss is 0.9493
+[ Info: Loss is 0.9647
+[ Info: Loss is 0.8598
+[ Info: Loss is 0.8699
+[ Info: Loss is 0.859
+[ Info: Loss is 0.843
+[ Info: Loss is 0.8556
+[ Info: Loss is 0.8016
+[ Info: Loss is 0.7762
+[ Info: Loss is 0.7672
+[ Info: Loss is 0.6868
+[ Info: Loss is 0.7414
+[ Info: Loss is 0.7573
+[ Info: Loss is 0.7062
+[ Info: Loss is 0.7683
+[ Info: Loss is 0.725
+[ Info: Loss is 0.8516
+[ Info: Loss is 0.7587
+[ Info: Loss is 0.7132
````
@@ -827,7 +827,7 @@ yhat[1]
````
````
-UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.155, Iris-versicolor=>0.582, Iris-virginica=>0.262)
+UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.156, Iris-versicolor=>0.519, Iris-virginica=>0.326)
````
What's going on here?
@@ -858,7 +858,7 @@ pdf(yhat[1], "Iris-virginica")
````
````
-0.26227638f0
+0.32571402f0
````
To get the most likely observation, we do
@@ -879,10 +879,10 @@ broadcast(pdf, yhat[1:4], "Iris-versicolor")
````
4-element Vector{Float32}:
- 0.58242285
- 0.03553577
- 0.03338368
- 0.035628445
+ 0.5187301
+ 0.0025167037
+ 0.00241053
+ 0.0027305766
````
````@julia
@@ -921,10 +921,10 @@ pdf(yhat, L)[1:4, :]
````
4×3 Matrix{Float32}:
- 0.155301 0.582423 0.262276
- 0.963971 0.0355358 0.000493214
- 0.966181 0.0333837 0.000435089
- 0.963876 0.0356284 0.000495223
+ 0.155556 0.51873 0.325714
+ 0.997162 0.0025167 0.000321372
+ 0.997287 0.00241053 0.000302146
+ 0.996915 0.00273058 0.000354911
````
However, in a typical MLJ workflow, this is not as useful as you might imagine. In
@@ -936,7 +936,7 @@ log_loss(yhat, y[test])
````
````
-0.3571473942077973
+0.41664216811287624
````
To apply a deterministic measure, we first need to obtain point-estimates:
@@ -946,7 +946,7 @@ misclassification_rate(mode.(yhat), y[test])
````
````
-0.044444444444444446
+0.08888888888888889
````
For more on metrics provided by MLJ, see the [StatisticalMeasures.jl
@@ -970,20 +970,21 @@ evaluate!(
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: NeuralNetworkClassifier-481
+Tag: NeuralNetworkClassifier-433
Extract:
┌─────────────────────────┬──────────────┬─────────────┐
│ measure │ operation │ measurement │
├─────────────────────────┼──────────────┼─────────────┤
-│ LogLoss( │ predict │ 0.357 │
+│ LogLoss( │ predict │ 0.417 │
│ tol = 2.22045e-16) │ │ │
-│ MisclassificationRate() │ predict_mode │ 0.0444 │
-│ BrierScore() │ predict │ -0.187 │
+│ MisclassificationRate() │ predict_mode │ 0.0889 │
+│ BrierScore() │ predict │ -0.236 │
└─────────────────────────┴──────────────┴─────────────┘
-
+Apply `describe` to this result for a named tuple summary.
````
Or applying cross-validation instead:
@@ -999,27 +1000,28 @@ evaluate!(
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: NeuralNetworkClassifier-855
+Tag: NeuralNetworkClassifier-136
Extract:
-┌───┬─────────────────────────┬──────────────┬─────────────┐
-│ │ measure │ operation │ measurement │
-├───┼─────────────────────────┼──────────────┼─────────────┤
-│ A │ LogLoss( │ predict │ 0.319 │
-│ │ tol = 2.22045e-16) │ │ │
-│ B │ MisclassificationRate() │ predict_mode │ 0.04 │
-│ C │ BrierScore() │ predict │ -0.168 │
-└───┴─────────────────────────┴──────────────┴─────────────┘
-┌───┬───────────────────────────────────────────────────────┬─────────┐
-│ │ per_fold │ 1.96*SE │
-├───┼───────────────────────────────────────────────────────┼─────────┤
-│ A │ [0.363, 0.271, 0.265, 0.305, 0.32, 0.388] │ 0.0432 │
-│ B │ [0.04, 0.04, 0.0, 0.08, 0.04, 0.04] │ 0.0222 │
-│ C │ Float32[-0.205, -0.135, -0.139, -0.17, -0.15, -0.206] │ 0.0279 │
-└───┴───────────────────────────────────────────────────────┴─────────┘
-
+┌───┬─────────────────────────┬──────────────┬─────────────┬─────────┐
+│ │ measure │ operation │ measurement │ 1.96*SE │
+├───┼─────────────────────────┼──────────────┼─────────────┼─────────┤
+│ A │ LogLoss( │ predict │ 0.276 │ 0.035 │
+│ │ tol = 2.22045e-16) │ │ │ │
+│ B │ MisclassificationRate() │ predict_mode │ 0.04 │ 0.022 │
+│ C │ BrierScore() │ predict │ -0.143 │ 0.026 │
+└───┴─────────────────────────┴──────────────┴─────────────┴─────────┘
+┌───┬──────────────────────────────────────────────────────────┐
+│ │ per_fold │
+├───┼──────────────────────────────────────────────────────────┤
+│ A │ [0.312, 0.256, 0.219, 0.263, 0.277, 0.33] │
+│ B │ [0.04, 0.04, 0.0, 0.04, 0.08, 0.04] │
+│ C │ Float32[-0.181, -0.127, -0.0987, -0.137, -0.142, -0.172] │
+└───┴──────────────────────────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
Or, Monte Carlo cross-validation (cross-validation with repeated
@@ -1037,28 +1039,29 @@ e = evaluate!(
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: NeuralNetworkClassifier-990
+Tag: NeuralNetworkClassifier-762
Extract:
-┌───┬─────────────────────────┬──────────────┬─────────────┐
-│ │ measure │ operation │ measurement │
-├───┼─────────────────────────┼──────────────┼─────────────┤
-│ A │ LogLoss( │ predict │ 0.316 │
-│ │ tol = 2.22045e-16) │ │ │
-│ B │ MisclassificationRate() │ predict_mode │ 0.0422 │
-│ C │ BrierScore() │ predict │ -0.172 │
-└───┴─────────────────────────┴──────────────┴─────────────┘
+┌───┬─────────────────────────┬──────────────┬─────────────┬─────────┐
+│ │ measure │ operation │ measurement │ 1.96*SE │
+├───┼─────────────────────────┼──────────────┼─────────────┼─────────┤
+│ A │ LogLoss( │ predict │ 0.318 │ 0.035 │
+│ │ tol = 2.22045e-16) │ │ │ │
+│ B │ MisclassificationRate() │ predict_mode │ 0.042 │ 0.023 │
+│ C │ BrierScore() │ predict │ -0.171 │ 0.023 │
+└───┴─────────────────────────┴──────────────┴─────────────┴─────────┘
┌───┬───────────────────────────────────────────────────────────────────────────
│ │ per_fold ⋯
├───┼───────────────────────────────────────────────────────────────────────────
-│ A │ [0.363, 0.328, 0.281, 0.313, 0.355, 0.22, 0.29, 0.403, 0.309, 0.386, 0.4 ⋯
-│ B │ [0.0, 0.0, 0.04, 0.0, 0.08, 0.0, 0.04, 0.12, 0.0, 0.04, 0.12, 0.08, 0.08 ⋯
-│ C │ Float32[-0.21, -0.181, -0.144, -0.152, -0.208, -0.103, -0.164, -0.257, - ⋯
+│ A │ [0.263, 0.309, 0.294, 0.313, 0.317, 0.216, 0.388, 0.297, 0.427, 0.275, 0 ⋯
+│ B │ [0.0, 0.04, 0.04, 0.0, 0.04, 0.04, 0.2, 0.0, 0.0, 0.04, 0.04, 0.08, 0.08 ⋯
+│ C │ Float32[-0.133, -0.164, -0.153, -0.156, -0.175, -0.117, -0.246, -0.158, ⋯
└───┴───────────────────────────────────────────────────────────────────────────
- 2 columns omitted
-
+ 1 column omitted
+Apply `describe` to this result for a named tuple summary.
````
We finally note that you can restrict the rows of observations from
@@ -1082,51 +1085,51 @@ predict(mach, rows=test) # and predict missing targets
````
45-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, String, UInt32, Float32}:
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.212, Iris-versicolor=>0.565, Iris-virginica=>0.223)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.989, Iris-versicolor=>0.0114, Iris-virginica=>4.52e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.988, Iris-versicolor=>0.0125, Iris-virginica=>5.2e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.986, Iris-versicolor=>0.0142, Iris-virginica=>6.58e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.236, Iris-versicolor=>0.564, Iris-virginica=>0.2)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00401, Iris-versicolor=>0.257, Iris-virginica=>0.739)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.99, Iris-versicolor=>0.00983, Iris-virginica=>3.51e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.989, Iris-versicolor=>0.0115, Iris-virginica=>4.51e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0575, Iris-versicolor=>0.445, Iris-virginica=>0.497)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.3, Iris-versicolor=>0.612, Iris-virginica=>0.0877)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00131, Iris-versicolor=>0.194, Iris-virginica=>0.805)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00405, Iris-versicolor=>0.259, Iris-virginica=>0.737)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00183, Iris-versicolor=>0.21, Iris-virginica=>0.788)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.15, Iris-versicolor=>0.514, Iris-virginica=>0.336)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.988, Iris-versicolor=>0.0124, Iris-virginica=>5.19e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.333, Iris-versicolor=>0.615, Iris-virginica=>0.052)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00342, Iris-versicolor=>0.243, Iris-virginica=>0.754)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00366, Iris-versicolor=>0.249, Iris-virginica=>0.748)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0363, Iris-versicolor=>0.416, Iris-virginica=>0.548)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.209, Iris-versicolor=>0.555, Iris-virginica=>0.236)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.392, Iris-versicolor=>0.584, Iris-virginica=>0.024)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.171, Iris-versicolor=>0.546, Iris-virginica=>0.282)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.225, Iris-versicolor=>0.589, Iris-virginica=>0.186)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00249, Iris-versicolor=>0.225, Iris-virginica=>0.772)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.983, Iris-versicolor=>0.017, Iris-virginica=>8.92e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00176, Iris-versicolor=>0.208, Iris-virginica=>0.79)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.262, Iris-versicolor=>0.597, Iris-virginica=>0.14)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0433, Iris-versicolor=>0.405, Iris-virginica=>0.552)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00694, Iris-versicolor=>0.287, Iris-virginica=>0.706)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00519, Iris-versicolor=>0.269, Iris-virginica=>0.726)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.161, Iris-versicolor=>0.568, Iris-virginica=>0.271)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.984, Iris-versicolor=>0.0164, Iris-virginica=>8.65e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.176, Iris-versicolor=>0.569, Iris-virginica=>0.255)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.236, Iris-versicolor=>0.593, Iris-virginica=>0.17)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00228, Iris-versicolor=>0.222, Iris-virginica=>0.776)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.04, Iris-versicolor=>0.419, Iris-virginica=>0.541)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00363, Iris-versicolor=>0.249, Iris-virginica=>0.748)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.302, Iris-versicolor=>0.606, Iris-virginica=>0.0924)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.988, Iris-versicolor=>0.0124, Iris-virginica=>5.31e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.987, Iris-versicolor=>0.013, Iris-virginica=>5.6e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00524, Iris-versicolor=>0.271, Iris-virginica=>0.724)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.985, Iris-versicolor=>0.0155, Iris-virginica=>7.67e-8)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0264, Iris-versicolor=>0.39, Iris-virginica=>0.584)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.245, Iris-versicolor=>0.582, Iris-virginica=>0.173)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0246, Iris-versicolor=>0.38, Iris-virginica=>0.595)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0903, Iris-versicolor=>0.528, Iris-virginica=>0.382)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.979, Iris-versicolor=>0.0153, Iris-virginica=>0.00547)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.978, Iris-versicolor=>0.0163, Iris-virginica=>0.00586)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.975, Iris-versicolor=>0.0185, Iris-virginica=>0.00674)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.134, Iris-versicolor=>0.524, Iris-virginica=>0.342)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>8.56e-5, Iris-versicolor=>0.228, Iris-virginica=>0.772)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.983, Iris-versicolor=>0.0127, Iris-virginica=>0.00446)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.98, Iris-versicolor=>0.0149, Iris-virginica=>0.00533)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00347, Iris-versicolor=>0.395, Iris-virginica=>0.602)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.135, Iris-versicolor=>0.521, Iris-virginica=>0.343)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>2.86e-5, Iris-versicolor=>0.19, Iris-virginica=>0.81)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>9.2e-5, Iris-versicolor=>0.231, Iris-virginica=>0.769)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>3.58e-5, Iris-versicolor=>0.198, Iris-virginica=>0.802)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0527, Iris-versicolor=>0.52, Iris-virginica=>0.427)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.978, Iris-versicolor=>0.0164, Iris-virginica=>0.0059)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.144, Iris-versicolor=>0.518, Iris-virginica=>0.338)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>8.25e-5, Iris-versicolor=>0.227, Iris-virginica=>0.773)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.000144, Iris-versicolor=>0.248, Iris-virginica=>0.752)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00414, Iris-versicolor=>0.404, Iris-virginica=>0.592)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0957, Iris-versicolor=>0.528, Iris-virginica=>0.376)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.215, Iris-versicolor=>0.494, Iris-virginica=>0.291)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.045, Iris-versicolor=>0.515, Iris-virginica=>0.44)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0742, Iris-versicolor=>0.526, Iris-virginica=>0.4)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>5.45e-5, Iris-versicolor=>0.212, Iris-virginica=>0.788)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.964, Iris-versicolor=>0.0261, Iris-virginica=>0.0098)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>4.02e-5, Iris-versicolor=>0.202, Iris-virginica=>0.798)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.131, Iris-versicolor=>0.523, Iris-virginica=>0.346)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00176, Iris-versicolor=>0.361, Iris-virginica=>0.637)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.000152, Iris-versicolor=>0.25, Iris-virginica=>0.749)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.000158, Iris-versicolor=>0.252, Iris-virginica=>0.748)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0393, Iris-versicolor=>0.51, Iris-virginica=>0.451)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.969, Iris-versicolor=>0.023, Iris-virginica=>0.00853)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0425, Iris-versicolor=>0.513, Iris-virginica=>0.445)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.097, Iris-versicolor=>0.527, Iris-virginica=>0.376)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>4.55e-5, Iris-versicolor=>0.206, Iris-virginica=>0.794)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00276, Iris-versicolor=>0.383, Iris-virginica=>0.614)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>9.93e-5, Iris-versicolor=>0.234, Iris-virginica=>0.766)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.138, Iris-versicolor=>0.521, Iris-virginica=>0.341)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.977, Iris-versicolor=>0.0165, Iris-virginica=>0.00596)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.977, Iris-versicolor=>0.0171, Iris-virginica=>0.00619)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.000143, Iris-versicolor=>0.248, Iris-virginica=>0.752)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.97, Iris-versicolor=>0.0217, Iris-virginica=>0.00804)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00164, Iris-versicolor=>0.357, Iris-virginica=>0.641)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.124, Iris-versicolor=>0.525, Iris-virginica=>0.351)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00131, Iris-versicolor=>0.346, Iris-virginica=>0.652)
````
### On learning curves
@@ -1155,7 +1158,7 @@ curve = learning_curve(
````
````
-(parameter_name = "epochs", parameter_scale = :log10, parameter_values = [1, 2, 3, 4, 5, 7, 9, 11, 14, 17, 22, 28, 36, 45, 57, 73, 92, 117, 149, 189, 240, 304, 386, 489, 621, 788, 1000], measurements = [0.996922160303613, 0.8577055250248313, 0.7981519638729986, 0.7620389845823741, 0.7323274239258524, 0.6767194293543304, 0.6456183342973714, 0.6019803869670477, 0.6148576306918716, 0.5858385363538815, 0.5420341283852436, 0.5502000369070692, 0.5408171567564948, 0.485989693319718, 0.4936070614522306, 0.4330706602760043, 0.4150256223224028, 0.3341514384124167, 0.2983904685917199, 0.2815979750419263, 0.27383412873432805, 0.2781026630466714, 0.2515121244169944, 0.25689560519828347, 0.2283805057524257, 0.2597360768185133, 0.24229163132135162])
+(parameter_name = "epochs", parameter_scale = :log10, parameter_values = [1, 2, 3, 4, 5, 7, 9, 11, 14, 17, 22, 28, 36, 45, 57, 73, 92, 117, 149, 189, 240, 304, 386, 489, 621, 788, 1000], measurements = [0.9792367389813159, 0.861137424251525, 0.795194663958999, 0.7178450923341597, 0.6672852524240639, 0.6298719732484978, 0.5722341032114211, 0.5468304476118365, 0.5176573197034343, 0.49085053804968687, 0.4656446207911134, 0.4495242488591051, 0.4099614796325163, 0.42257271926898354, 0.31717065032020536, 0.3321625563432286, 0.2845664781923225, 0.22536592494730678, 0.1853560701976897, 0.24943364437705945, 0.24673236950764432, 0.20234975793565377, 0.20691858169733499, 0.20911653361243762, 0.22259957311318937, 0.21965174517509872, 0.21279597941067163])
````
````@julia
@@ -1218,16 +1221,16 @@ y4 = [n_devices(row.salary) for row in eachrow(X4)]
````
10-element Vector{Int64}:
+ 3
1
- 1
- 2
- 1
- 2
+ 3
0
+ 2
+ 6
0
0
+ 2
1
- 4
````
(b) What models can be applied if you coerce the salary to a
@@ -1253,10 +1256,10 @@ pretty(data)
│ Int64 │ Float64 │ Float64 │ CategoricalValue{String, UInt32} │
│ Count │ Continuous │ Continuous │ OrderedFactor{2} │
├───────┼────────────┼────────────┼──────────────────────────────────┤
-│ 1 │ 0.303334 │ 0.803837 │ male │
-│ 2 │ 0.765226 │ 0.061673 │ female │
-│ 3 │ 0.88446 │ 0.716328 │ female │
-│ 4 │ 0.255475 │ 0.239904 │ male │
+│ 1 │ 0.754755 │ 0.208445 │ male │
+│ 2 │ 0.874161 │ 0.399919 │ female │
+│ 3 │ 0.906202 │ 0.19441 │ female │
+│ 4 │ 0.718781 │ 0.818259 │ male │
└───────┴────────────┴────────────┴──────────────────────────────────┘
````
diff --git a/docs/src/notebooks/MLJTutorial/03_pipelines/notebook.md b/docs/src/notebooks/MLJTutorial/03_pipelines/notebook.md
index 4f4cbf6..f8049ef 100644
--- a/docs/src/notebooks/MLJTutorial/03_pipelines/notebook.md
+++ b/docs/src/notebooks/MLJTutorial/03_pipelines/notebook.md
@@ -31,8 +31,8 @@ x = rand(100);
````
````
-mean(x) = 0.52573272422357
-std(x) = 0.3020501201265377
+mean(x) = 0.47696749924138343
+std(x) = 0.29728842148044776
````
@@ -46,7 +46,7 @@ xhat = transform(mach, x);
````
[ Info: Training machine(Standardizer(features = Symbol[], …), …).
-mean(xhat) = -1.8596235662471373e-16
+mean(xhat) = 2.042810365310288e-16
std(xhat) = 1.0
````
@@ -497,10 +497,11 @@ evaluate!(mach, measure=mae, resampling=Holdout()) # `CV(nfolds=6)` is `resampli
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicPipeline-661
+Tag: DeterministicPipeline-190
Extract:
┌──────────┬───────────┬─────────────┐
│ measure │ operation │ measurement │
@@ -508,7 +509,7 @@ Extract:
│ LPLoss( │ predict │ 176000.0 │
│ p = 1) │ │ │
└──────────┴───────────┴─────────────┘
-
+Apply `describe` to this result for a named tuple summary.
````
### Training of composite models is "smart"
@@ -626,23 +627,24 @@ evaluate!(mach, measure=mae)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicPipeline-106
+Tag: DeterministicPipeline-398
Extract:
-┌──────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 162000.0 │
-│ p = 1) │ │ │
-└──────────┴───────────┴─────────────┘
-┌──────────────────────────────────────────────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├──────────────────────────────────────────────────────────────┼─────────┤
-│ [160000.0, 170000.0, 163000.0, 156000.0, 163000.0, 162000.0] │ 4140.0 │
-└──────────────────────────────────────────────────────────────┴─────────┘
-
+┌──────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼───────────┼─────────────┼─────────┤
+│ LPLoss( │ predict │ 162300.0 │ 4100.0 │
+│ p = 1) │ │ │ │
+└──────────┴───────────┴─────────────┴─────────┘
+┌──────────────────────────────────────────────────────────────┐
+│ per_fold │
+├──────────────────────────────────────────────────────────────┤
+│ [160000.0, 170000.0, 163000.0, 156000.0, 163000.0, 162000.0] │
+└──────────────────────────────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
MLJ will also allow you to insert *learned* target transformations. For example, we
@@ -666,23 +668,24 @@ evaluate!(mach, measure=mae)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicPipeline-264
+Tag: DeterministicPipeline-611
Extract:
-┌──────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 509000.0 │
-│ p = 1) │ │ │
-└──────────┴───────────┴─────────────┘
-┌───────────────────────────────────────────────────────────┬──────────┐
-│ per_fold │ 1.96*SE │
-├───────────────────────────────────────────────────────────┼──────────┤
-│ [162000.0, 2.2e6, 181000.0, 161000.0, 176000.0, 172000.0] │ 728000.0 │
-└───────────────────────────────────────────────────────────┴──────────┘
-
+┌──────────┬───────────┬─────────────┬──────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼───────────┼─────────────┼──────────┤
+│ LPLoss( │ predict │ 510000.0 │ 730000.0 │
+│ p = 1) │ │ │ │
+└──────────┴───────────┴─────────────┴──────────┘
+┌───────────────────────────────────────────────────────────┐
+│ per_fold │
+├───────────────────────────────────────────────────────────┤
+│ [162000.0, 2.2e6, 181000.0, 161000.0, 176000.0, 172000.0] │
+└───────────────────────────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
````@julia
@@ -693,23 +696,24 @@ evaluate!(mach, measure=mae)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicPipeline-300
+Tag: DeterministicPipeline-592
Extract:
-┌──────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 172000.0 │
-│ p = 1) │ │ │
-└──────────┴───────────┴─────────────┘
-┌──────────────────────────────────────────────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├──────────────────────────────────────────────────────────────┼─────────┤
-│ [171000.0, 172000.0, 173000.0, 170000.0, 173000.0, 171000.0] │ 1240.0 │
-└──────────────────────────────────────────────────────────────┴─────────┘
-
+┌──────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼───────────┼─────────────┼─────────┤
+│ LPLoss( │ predict │ 171500.0 │ 1200.0 │
+│ p = 1) │ │ │ │
+└──────────┴───────────┴─────────────┴─────────┘
+┌──────────────────────────────────────────────────────────────┐
+│ per_fold │
+├──────────────────────────────────────────────────────────────┤
+│ [171000.0, 172000.0, 173000.0, 170000.0, 173000.0, 171000.0] │
+└──────────────────────────────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
### Tutorial 3 Resources
diff --git a/docs/src/notebooks/MLJTutorial/04_tuning/gamma_sampler.png b/docs/src/notebooks/MLJTutorial/04_tuning/gamma_sampler.png
index 253f458..8ddb87e 100644
Binary files a/docs/src/notebooks/MLJTutorial/04_tuning/gamma_sampler.png and b/docs/src/notebooks/MLJTutorial/04_tuning/gamma_sampler.png differ
diff --git a/docs/src/notebooks/MLJTutorial/04_tuning/notebook.md b/docs/src/notebooks/MLJTutorial/04_tuning/notebook.md
index 990c886..9882163 100644
--- a/docs/src/notebooks/MLJTutorial/04_tuning/notebook.md
+++ b/docs/src/notebooks/MLJTutorial/04_tuning/notebook.md
@@ -363,23 +363,24 @@ err = evaluate!(mach, resampling=CV(nfolds=3), measure=log_loss)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: ProbabilisticPipeline-509
+Tag: ProbabilisticPipeline-307
Extract:
-┌──────────────────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────────────────┼───────────┼─────────────┤
-│ LogLoss( │ predict │ 0.782 │
-│ tol = 2.22045e-16) │ │ │
-└──────────────────────┴───────────┴─────────────┘
-┌──────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├──────────────────────┼─────────┤
-│ [0.83, 0.721, 0.794] │ 0.0773 │
-└──────────────────────┴─────────┘
-
+┌──────────────────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────────────────┼───────────┼─────────────┼─────────┤
+│ LogLoss( │ predict │ 0.782 │ 0.077 │
+│ tol = 2.22045e-16) │ │ │ │
+└──────────────────────┴───────────┴─────────────┴─────────┘
+┌──────────────────────┐
+│ per_fold │
+├──────────────────────┤
+│ [0.83, 0.721, 0.794] │
+└──────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
````@julia
@@ -389,23 +390,24 @@ tuned_err = evaluate!(tuned_mach, resampling=CV(nfolds=3), measure=log_loss)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: ProbabilisticTunedModel-395
+Tag: ProbabilisticTunedModel-779
Extract:
-┌──────────────────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────────────────┼───────────┼─────────────┤
-│ LogLoss( │ predict │ 0.779 │
-│ tol = 2.22045e-16) │ │ │
-└──────────────────────┴───────────┴─────────────┘
-┌───────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├───────────────────────┼─────────┤
-│ [0.798, 0.802, 0.738] │ 0.0496 │
-└───────────────────────┴─────────┘
-
+┌──────────────────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────────────────┼───────────┼─────────────┼─────────┤
+│ LogLoss( │ predict │ 0.779 │ 0.05 │
+│ tol = 2.22045e-16) │ │ │ │
+└──────────────────────┴───────────┴─────────────┴─────────┘
+┌───────────────────────┐
+│ per_fold │
+├───────────────────────┤
+│ [0.798, 0.802, 0.738] │
+└───────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
### Tutorial 4 Resources
diff --git a/docs/src/notebooks/MLJTutorial/05_composition/notebook.md b/docs/src/notebooks/MLJTutorial/05_composition/notebook.md
index f4a1692..12dbe0a 100644
--- a/docs/src/notebooks/MLJTutorial/05_composition/notebook.md
+++ b/docs/src/notebooks/MLJTutorial/05_composition/notebook.md
@@ -64,11 +64,11 @@ pretty(X)
│ Float64 │ Float64 │ Float64 │
│ Continuous │ Continuous │ Continuous │
├────────────┼────────────┼────────────┤
-│ 5.31563 │ -3.3673 │ -10.3947 │
-│ -0.249305 │ 4.31778 │ 5.53754 │
-│ 5.32621 │ 11.1213 │ -4.48258 │
-│ 1.26847 │ 6.21007 │ 4.83913 │
-│ -0.599664 │ 5.77754 │ 4.42261 │
+│ 14.9681 │ -1.47137 │ 12.1949 │
+│ -3.26626 │ -0.710617 │ 7.36507 │
+│ -4.17429 │ -0.0882215 │ 9.87907 │
+│ 6.00034 │ -2.41859 │ 14.9897 │
+│ -4.38537 │ -0.187352 │ 7.22233 │
└────────────┴────────────┴────────────┘
````
@@ -91,11 +91,11 @@ yhat = predict(mach2, Xstand)
````
5-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, Int64, UInt32, Float64}:
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.996, 2=>0.00197, 3=>0.00173)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000517, 2=>0.000196, 3=>0.999)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00154, 2=>0.994, 3=>0.00397)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00128, 2=>0.00405, 3=>0.995)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000393, 2=>0.000447, 3=>0.999)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.993, 2=>0.00372, 3=>0.00319)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000405, 2=>0.000195, 3=>0.999)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000309, 2=>0.000249, 3=>0.999)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00752, 2=>0.992, 3=>0.000947)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00012, 2=>1.96e-5, 3=>1.0)
````
**Step 1** - Edit your code as follows:
@@ -119,15 +119,15 @@ yhat = predict(mach2, Xstand)
````
````
-Node @316 → LogisticClassifier(…)
+Node @440 → LogisticClassifier(…)
args:
- 1: Node @206 → Standardizer(…)
+ 1: Node @909 → Standardizer(…)
formula:
predict(
machine(LogisticClassifier(lambda = 0.001, …), …),
transform(
machine(Standardizer(features = Symbol[], …), …),
- Source @679,
+ Source @128,
),
)
````
@@ -150,11 +150,11 @@ Xstand() |> pretty
│ Float64 │ Float64 │ Float64 │
│ Continuous │ Continuous │ Continuous │
├────────────┼────────────┼────────────┤
-│ 1.06157 │ -1.56085 │ -1.46133 │
-│ -0.84203 │ -0.0942899 │ 0.781855 │
-│ 1.06519 │ 1.20404 │ -0.62893 │
-│ -0.322846 │ 0.266821 │ 0.683523 │
-│ -0.961878 │ 0.18428 │ 0.624879 │
+│ 1.54143 │ -0.508564 │ 0.563239 │
+│ -0.597675 │ 0.271238 │ -0.895631 │
+│ -0.704197 │ 0.909215 │ -0.136268 │
+│ 0.489404 │ -1.47949 │ 1.40741 │
+│ -0.728959 │ 0.807603 │ -0.938747 │
└────────────┴────────────┴────────────┘
````
@@ -178,11 +178,11 @@ yhat()
````
5-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, Int64, UInt32, Float64}:
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.996, 2=>0.00197, 3=>0.00173)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000517, 2=>0.000196, 3=>0.999)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00154, 2=>0.994, 3=>0.00397)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00128, 2=>0.00405, 3=>0.995)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000393, 2=>0.000447, 3=>0.999)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.993, 2=>0.00372, 3=>0.00319)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000405, 2=>0.000195, 3=>0.999)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000309, 2=>0.000249, 3=>0.999)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00752, 2=>0.992, 3=>0.000947)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00012, 2=>1.96e-5, 3=>1.0)
````
The node `yhat` is the "descendant" (in an associated DAG we have
@@ -194,7 +194,7 @@ origins(yhat)
````
1-element Vector{MLJBase.Source}:
- Source @679 ⏎ `ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}}`
+ Source @128 ⏎ `ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}}`
````
The data at the source node is replaced by `Xnew` to obtain a
@@ -207,8 +207,8 @@ yhat(Xnew)
````
2-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, Int64, UInt32, Float64}:
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>2.72e-7, 2=>1.01e-7, 3=>1.0)
- UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>2.8e-6, 2=>8.96e-5, 3=>1.0)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>1.13e-8, 2=>1.0, 3=>6.09e-8)
+ UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>2.89e-16, 2=>1.0, 3=>4.36e-13)
````
**Step 2** - Export the learning network as a new stand-alone model type
@@ -238,15 +238,15 @@ yhat = predict(mach2, Xstand)
````
````
-Node @855 → :classifier
+Node @530 → :classifier
args:
- 1: Node @680 → :standardizer
+ 1: Node @804 → :standardizer
formula:
predict(
machine(:classifier, …),
transform(
machine(:standardizer, …),
- Source @679,
+ Source @128,
),
)
````
@@ -323,10 +323,10 @@ fitted_params(mach).classifier.coefs
````
4-element Vector{Pair{Symbol, SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}}}:
- :sepal_length => [-2.2884323728871254, 1.4607388496829115, 0.8276935232042247]
- :sepal_width => [2.5849165395349716, -0.6646830918858928, -1.9202334476490799]
- :petal_length => [-3.5313085896004583, -1.406180852406463, 4.937489442006912]
- :petal_width => [-3.4480076498857017, -1.6977190775004856, 5.145726727386192]
+ :sepal_length => [-2.2884323728871574, 1.4607388496829223, 0.8276935232042454]
+ :sepal_width => [2.5849165395350733, -0.6646830918859379, -1.9202334476491332]
+ :petal_length => [-3.5313085896005036, -1.4061808524064117, 4.937489442006905]
+ :petal_width => [-3.448007649885748, -1.6977190775004702, 5.145726727386221]
````
````@julia
@@ -380,7 +380,7 @@ y = source(y)
````
````
-Source @924 ⏎ `AbstractVector{ScientificTypesBase.Continuous}`
+Source @555 ⏎ `AbstractVector{ScientificTypesBase.Continuous}`
````
**First layer and target transformation:**
@@ -396,13 +396,13 @@ z = MLJ.transform(mach2, y)
````
````
-Node @546 → UnivariateBoxCoxTransformer(…)
+Node @022 → UnivariateBoxCoxTransformer(…)
args:
- 1: Source @924
+ 1: Source @555
formula:
transform(
machine(UnivariateBoxCoxTransformer(n = 171, …), …),
- Source @924,
+ Source @555,
)
````
@@ -419,10 +419,10 @@ zhat = 0.5*predict(mach3, W) + 0.5*predict(mach4, W)
````
````
-Node @505
+Node @135
args:
- 1: Node @084
- 2: Node @653
+ 1: Node @083
+ 2: Node @797
formula:
+(
var"#*##0#*##1"(
@@ -430,7 +430,7 @@ Node @505
machine(RidgeRegressor(lambda = 0.1, …), …),
transform(
machine(Standardizer(features = Symbol[], …), …),
- Source @511,
+ Source @100,
),
),
),
@@ -439,7 +439,7 @@ Node @505
machine(RandomForestRegressor(max_depth = -1, …), …),
transform(
machine(Standardizer(features = Symbol[], …), …),
- Source @511,
+ Source @100,
),
),
),
@@ -453,9 +453,9 @@ yhat = inverse_transform(mach2, zhat)
````
````
-Node @247 → UnivariateBoxCoxTransformer(…)
+Node @539 → UnivariateBoxCoxTransformer(…)
args:
- 1: Node @505
+ 1: Node @135
formula:
inverse_transform(
machine(UnivariateBoxCoxTransformer(n = 171, …), …),
@@ -465,7 +465,7 @@ Node @247 → UnivariateBoxCoxTransformer(…)
machine(RidgeRegressor(lambda = 0.1, …), …),
transform(
machine(Standardizer(features = Symbol[], …), …),
- Source @511,
+ Source @100,
),
),
),
@@ -474,7 +474,7 @@ Node @247 → UnivariateBoxCoxTransformer(…)
machine(RandomForestRegressor(max_depth = -1, …), …),
transform(
machine(Standardizer(features = Symbol[], …), …),
- Source @511,
+ Source @100,
),
),
),
@@ -491,9 +491,9 @@ yhat(rows=1:3)
````
3-element Vector{Float64}:
- 0.38643611843841796
- 0.5723985261534488
- 0.44151392790057636
+ 4.1049434111127105
+ 4.674958202525163
+ 4.112097582032496
````
Now for the new model type:
@@ -543,25 +543,26 @@ evaluate(composite, X, y; resampling=CV(nfolds=6, shuffle=true), measures=[rms,
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: CompositeModel-412
+Tag: CompositeModel-983
Extract:
-┌───┬────────────────────────┬───────────┬─────────────┐
-│ │ measure │ operation │ measurement │
-├───┼────────────────────────┼───────────┼─────────────┤
-│ A │ RootMeanSquaredError() │ predict │ 4.0 │
-│ B │ LPLoss( │ predict │ 2.51 │
-│ │ p = 1) │ │ │
-└───┴────────────────────────┴───────────┴─────────────┘
-┌───┬─────────────────────────────────────┬─────────┐
-│ │ per_fold │ 1.96*SE │
-├───┼─────────────────────────────────────┼─────────┤
-│ A │ [2.67, 2.9, 4.56, 3.85, 5.84, 3.28] │ 1.04 │
-│ B │ [1.81, 2.21, 2.81, 2.27, 3.6, 2.37] │ 0.547 │
-└───┴─────────────────────────────────────┴─────────┘
-
+┌───┬────────────────────────┬───────────┬─────────────┬─────────┐
+│ │ measure │ operation │ measurement │ 1.96*SE │
+├───┼────────────────────────┼───────────┼─────────────┼─────────┤
+│ A │ RootMeanSquaredError() │ predict │ 3.87 │ 0.46 │
+│ B │ LPLoss( │ predict │ 2.454 │ 0.04 │
+│ │ p = 1) │ │ │ │
+└───┴────────────────────────┴───────────┴─────────────┴─────────┘
+┌───┬──────────────────────────────────────┐
+│ │ per_fold │
+├───┼──────────────────────────────────────┤
+│ A │ [4.45, 4.28, 3.41, 3.3, 4.21, 3.37] │
+│ B │ [2.45, 2.46, 2.46, 2.38, 2.53, 2.45] │
+└───┴──────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
### Tutorial 5 Resources
diff --git a/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/exercise_7c.png b/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/exercise_7c.png
index 553fd96..08ac21b 100644
Binary files a/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/exercise_7c.png and b/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/exercise_7c.png differ
diff --git a/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/notebook.md b/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/notebook.md
index 7624a2d..5cca734 100644
--- a/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/notebook.md
+++ b/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/notebook.md
@@ -50,8 +50,8 @@ A = rand(2, 3)
````
2×3 Matrix{Float64}:
- 0.441502 0.871795 0.685263
- 0.535322 0.850152 0.76627
+ 0.301055 0.691879 0.504642
+ 0.572222 0.788816 0.738401
````
````@julia
@@ -77,8 +77,8 @@ Asparse = sparse(A)
````
2×3 SparseArrays.SparseMatrixCSC{Float64, Int64} with 6 stored entries:
- 0.441502 0.871795 0.685263
- 0.535322 0.850152 0.76627
+ 0.301055 0.691879 0.504642
+ 0.572222 0.788816 0.738401
````
````@julia
@@ -95,8 +95,8 @@ C = coerce(A, Multiclass)
````
2×3 CategoricalArrays.CategoricalArray{Float64,2,UInt32}:
- 0.441502 0.871795 0.685263
- 0.535322 0.850152 0.76627
+ 0.301055 0.691879 0.504642
+ 0.572222 0.788816 0.738401
````
````@julia
@@ -326,15 +326,15 @@ y4 = [n_devices(row.salary) for row in eachrow(X4)]
````
10-element Vector{Int64}:
- 4
- 3
2
- 4
- 6
- 1
3
2
+ 3
+ 3
+ 0
2
+ 1
+ 1
2
````
@@ -411,10 +411,10 @@ pretty(X)
│ Float64 │ Float64 │
│ Continuous │ Continuous │
├────────────┼────────────┤
-│ 0.256611 │ 0.672213 │
-│ 0.385614 │ 0.149492 │
-│ 0.873735 │ 0.269848 │
-│ 0.753635 │ 0.206412 │
+│ 0.11433 │ 0.538742 │
+│ 0.0704153 │ 0.19053 │
+│ 0.130344 │ 0.0186654 │
+│ 0.760926 │ 0.454443 │
└────────────┴────────────┘
````
@@ -518,7 +518,7 @@ fitted_params(mach)
````
````
-(classes = CategoricalArrays.CategoricalValue{Int64, UInt32}[CategoricalValue(CategoricalArrays.CategoricalPool{Int64, UInt32}([1, 2, 3]), 1), CategoricalValue(CategoricalArrays.CategoricalPool{Int64, UInt32}([1, 2, 3]), 2), CategoricalValue(CategoricalArrays.CategoricalPool{Int64, UInt32}([1, 2, 3]), 3)], coefs = Pair{Symbol, SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}}[:rectal_temperature => [0.01679940181352405, -0.006529534963799633, -0.010269866849724429], :pulse => [-0.002078922183617693, 0.00248508277505068, -0.00040616059143293954], :respiratory_rate => [-0.002078922183617693, 0.00248508277505068, -0.00040616059143293954], :packed_cell_volume => [0.0026345470622363147, 0.002707772252298626, -0.005342319314534946], :total_protein => [0.009110258500266177, -0.01679787903626504, 0.0076876205359988755]], intercept = [0.00043563961897106053, -0.00016706727792654562, -0.0005913867104289383])
+(classes = CategoricalArrays.CategoricalValue{Int64, UInt32}[CategoricalValue(CategoricalArrays.CategoricalPool{Int64, UInt32}([1, 2, 3]), 1), CategoricalValue(CategoricalArrays.CategoricalPool{Int64, UInt32}([1, 2, 3]), 2), CategoricalValue(CategoricalArrays.CategoricalPool{Int64, UInt32}([1, 2, 3]), 3)], coefs = Pair{Symbol, SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}}[:rectal_temperature => [0.016799401813523955, -0.0065295349637995905, -0.010269866849724361], :pulse => [-0.0020789221836177217, 0.0024850827750506855, -0.0004061605914329617], :respiratory_rate => [-0.0020789221836177217, 0.0024850827750506855, -0.0004061605914329617], :packed_cell_volume => [0.0026345470622364743, 0.002707772252298548, -0.005342319314535014], :total_protein => [0.009110258500266097, -0.016797879036265145, 0.0076876205359990195]], intercept = [0.00043563961897105766, -0.00016706727792654296, -0.0005913867104289052])
````
````@julia
@@ -532,7 +532,7 @@ err = log_loss(yhat, y[test])
````
````
-0.8334775485441969
+0.8334775485441962
````
6(b)(iii)
@@ -586,23 +586,24 @@ evaluate!(mach, resampling=CV(nfolds=6), measure=log_loss)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: RandomForestClassifier-372
+Tag: RandomForestClassifier-355
Extract:
-┌──────────────────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────────────────┼───────────┼─────────────┤
-│ LogLoss( │ predict │ 1.1 │
-│ tol = 2.22045e-16) │ │ │
-└──────────────────────┴───────────┴─────────────┘
-┌─────────────────────────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├─────────────────────────────────────────┼─────────┤
-│ [0.762, 1.39, 1.79, 1.31, 0.697, 0.629] │ 0.411 │
-└─────────────────────────────────────────┴─────────┘
-
+┌──────────────────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────────────────┼───────────┼─────────────┼─────────┤
+│ LogLoss( │ predict │ 1.27 │ 0.33 │
+│ tol = 2.22045e-16) │ │ │ │
+└──────────────────────┴───────────┴─────────────┴─────────┘
+┌──────────────────────────────────────┐
+│ per_fold │
+├──────────────────────────────────────┤
+│ [1.27, 1.4, 1.79, 1.27, 1.26, 0.616] │
+└──────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
````@julia
@@ -660,7 +661,7 @@ err_forest =
````
````
-0.987927309743752
+1.290842672700936
````
#### Exercise 7
@@ -722,23 +723,24 @@ evaluate!(mach, resampling=CV(nfolds=6), measure=log_loss)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: ProbabilisticPipeline-616
+Tag: ProbabilisticPipeline-524
Extract:
-┌──────────────────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────────────────┼───────────┼─────────────┤
-│ LogLoss( │ predict │ 0.847 │
-│ tol = 2.22045e-16) │ │ │
-└──────────────────────┴───────────┴─────────────┘
-┌───────────────────────────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├───────────────────────────────────────────┼─────────┤
-│ [0.888, 1.15, 0.847, 0.812, 0.782, 0.608] │ 0.154 │
-└───────────────────────────────────────────┴─────────┘
-
+┌──────────────────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────────────────┼───────────┼─────────────┼─────────┤
+│ LogLoss( │ predict │ 0.85 │ 0.11 │
+│ tol = 2.22045e-16) │ │ │ │
+└──────────────────────┴───────────┴─────────────┴─────────┘
+┌──────────────────────────────────────────┐
+│ per_fold │
+├──────────────────────────────────────────┤
+│ [0.927, 1.02, 0.773, 0.85, 0.875, 0.665] │
+└──────────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
7(c)
@@ -825,23 +827,24 @@ best_err = evaluate!(best_mach, resampling=CV(nfolds=3), measure=mae)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicPipeline-942
+Tag: DeterministicPipeline-491
Extract:
-┌──────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 66800.0 │
-│ p = 1) │ │ │
-└──────────┴───────────┴─────────────┘
-┌─────────────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├─────────────────────────────┼─────────┤
-│ [66200.0, 66700.0, 67500.0] │ 941.0 │
-└─────────────────────────────┴─────────┘
-
+┌──────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼───────────┼─────────────┼─────────┤
+│ LPLoss( │ predict │ 66780.0 │ 940.0 │
+│ p = 1) │ │ │ │
+└──────────┴───────────┴─────────────┴─────────┘
+┌─────────────────────────────┐
+│ per_fold │
+├─────────────────────────────┤
+│ [66200.0, 66700.0, 67500.0] │
+└─────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
````@julia
@@ -851,23 +854,24 @@ tuned_err = evaluate!(tuned_mach, resampling=CV(nfolds=3), measure=mae)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicTunedModel-175
+Tag: DeterministicTunedModel-859
Extract:
-┌──────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 67600.0 │
-│ p = 1) │ │ │
-└──────────┴───────────┴─────────────┘
-┌─────────────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├─────────────────────────────┼─────────┤
-│ [67600.0, 67800.0, 67500.0] │ 214.0 │
-└─────────────────────────────┴─────────┘
-
+┌──────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼───────────┼─────────────┼─────────┤
+│ LPLoss( │ predict │ 67620.0 │ 210.0 │
+│ p = 1) │ │ │ │
+└──────────┴───────────┴─────────────┴─────────┘
+┌─────────────────────────────┐
+│ per_fold │
+├─────────────────────────────┤
+│ [67600.0, 67800.0, 67500.0] │
+└─────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
---
diff --git a/docs/src/notebooks/UsingMLJ/01_basics/notebook.md b/docs/src/notebooks/UsingMLJ/01_basics/notebook.md
index 74a57b8..afb15fd 100644
--- a/docs/src/notebooks/UsingMLJ/01_basics/notebook.md
+++ b/docs/src/notebooks/UsingMLJ/01_basics/notebook.md
@@ -21,11 +21,12 @@ Pkg.status()
````
Status `~/work/MLJTutorial.jl/MLJTutorial.jl/docs/src/notebooks/UsingMLJ/01_basics/Project.toml`
- [336ed68f] CSV v0.10.17
+⌃ [336ed68f] CSV v0.10.17
[add582a8] MLJ v0.23.3
[c6f25543] MLJDecisionTreeInterface v0.5.0
- [b8865327] UnicodePlots v3.8.4
+ [b8865327] UnicodePlots v3.9.0
[44cfe95a] Pkg v1.12.1
+Info Packages marked with ⌃ have new versions available and may be upgradable.
````
@@ -186,8 +187,8 @@ fitted_params(mach)
````
(forest = Ensemble of Decision Trees
Trees: 100
-Avg Leaves: 147.07
-Avg Depth: 14.67,)
+Avg Leaves: 147.25
+Avg Depth: 14.49,)
````
````@julia
@@ -206,9 +207,9 @@ predict(mach, X)[1:3]
````
3-element Vector{Float64}:
- 26.394000000000002
- 22.337
- 34.722
+ 26.984
+ 22.682999999999993
+ 35.31
````
Predict in the `test` rows:
@@ -224,7 +225,7 @@ mae(ypred, y[test])
````
````
-4.623326732673267
+4.410905940594058
````
`mae` is actually just an alias:
@@ -267,19 +268,20 @@ evaluate!(mach; resampling=[(train, test),], measures=[mae, RSquared()])
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: RandomForestRegressor-413
+Tag: RandomForestRegressor-162
Extract:
┌────────────┬───────────┬─────────────┐
│ measure │ operation │ measurement │
├────────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 4.62 │
+│ LPLoss( │ predict │ 4.41 │
│ p = 1) │ │ │
-│ RSquared() │ predict │ 0.323 │
+│ RSquared() │ predict │ 0.369 │
└────────────┴───────────┴─────────────┘
-
+Apply `describe` to this result for a named tuple summary.
````
Something fancier:
@@ -295,25 +297,26 @@ evaluate(
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: RandomForestRegressor-666
+Tag: RandomForestRegressor-267
Extract:
-┌───┬────────────┬───────────┬─────────────┐
-│ │ measure │ operation │ measurement │
-├───┼────────────┼───────────┼─────────────┤
-│ A │ LPLoss( │ predict │ 3.0 │
-│ │ p = 1) │ │ │
-│ B │ RSquared() │ predict │ 0.656 │
-└───┴────────────┴───────────┴─────────────┘
-┌───┬──────────────────────────────────────────┬─────────┐
-│ │ per_fold │ 1.96*SE │
-├───┼──────────────────────────────────────────┼─────────┤
-│ A │ [2.24, 2.29, 3.32, 2.41, 4.73, 3.03] │ 0.834 │
-│ B │ [0.726, 0.838, 0.7, 0.819, 0.472, 0.376] │ 0.166 │
-└───┴──────────────────────────────────────────┴─────────┘
-
+┌───┬────────────┬───────────┬─────────────┬─────────┐
+│ │ measure │ operation │ measurement │ 1.96*SE │
+├───┼────────────┼───────────┼─────────────┼─────────┤
+│ A │ LPLoss( │ predict │ 2.99 │ 0.92 │
+│ │ p = 1) │ │ │ │
+│ B │ RSquared() │ predict │ 0.67 │ 0.17 │
+└───┴────────────┴───────────┴─────────────┴─────────┘
+┌───┬──────────────────────────────────────────┐
+│ │ per_fold │
+├───┼──────────────────────────────────────────┤
+│ A │ [2.14, 2.11, 3.32, 2.5, 4.9, 2.96] │
+│ B │ [0.74, 0.871, 0.73, 0.813, 0.454, 0.392] │
+└───┴──────────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
Dietterich's 5 x 2 test:
@@ -331,26 +334,26 @@ e = evaluate(
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: RandomForestRegressor-199
+Tag: RandomForestRegressor-760
Extract:
-┌───┬────────────┬───────────┬─────────────┐
-│ │ measure │ operation │ measurement │
-├───┼────────────┼───────────┼─────────────┤
-│ A │ LPLoss( │ predict │ 2.45 │
-│ │ p = 1) │ │ │
-│ B │ RSquared() │ predict │ 0.842 │
-└───┴────────────┴───────────┴─────────────┘
-┌───┬───────────────────────────────────────────────────────────────────────┬───
-│ │ per_fold │ ⋯
-├───┼───────────────────────────────────────────────────────────────────────┼───
-│ A │ [2.49, 2.49, 2.6, 2.47, 2.35, 2.52, 2.43, 2.35, 2.52, 2.24] │ ⋯
-│ B │ [0.817, 0.855, 0.835, 0.83, 0.853, 0.808, 0.863, 0.856, 0.835, 0.867] │ ⋯
-└───┴───────────────────────────────────────────────────────────────────────┴───
- 1 column omitted
-
+┌───┬────────────┬───────────┬─────────────┬─────────┐
+│ │ measure │ operation │ measurement │ 1.96*SE │
+├───┼────────────┼───────────┼─────────────┼─────────┤
+│ A │ LPLoss( │ predict │ 2.49 │ 0.14 │
+│ │ p = 1) │ │ │ │
+│ B │ RSquared() │ predict │ 0.823 │ 0.033 │
+└───┴────────────┴───────────┴─────────────┴─────────┘
+┌───┬────────────────────────────────────────────────────────────────────────┐
+│ │ per_fold │
+├───┼────────────────────────────────────────────────────────────────────────┤
+│ A │ [2.64, 2.35, 2.17, 2.66, 2.86, 2.27, 2.67, 2.47, 2.54, 2.28] │
+│ B │ [0.752, 0.874, 0.897, 0.758, 0.777, 0.853, 0.835, 0.792, 0.824, 0.864] │
+└───┴────────────────────────────────────────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
````@julia
@@ -359,8 +362,8 @@ e.uncertainty_radius_95
````
2-element Vector{Float64}:
- 0.06904375445230268
- 0.013098268581533108
+ 0.14383105341435692
+ 0.032989951374955834
````
# Interlude on scientific types
@@ -605,8 +608,8 @@ first(yprob, 5)
5-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{2}, InlineStrings.String7, UInt32, Float64}:
UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>1.0, >50K=>0.0)
UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>1.0, >50K=>0.0)
- UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.98, >50K=>0.02)
- UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.24, >50K=>0.76)
+ UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.91, >50K=>0.09)
+ UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.33, >50K=>0.67)
UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>1.0, >50K=>0.0)
````
@@ -617,8 +620,8 @@ yprob[3]
````
UnivariateFinite{ScientificTypesBase.Multiclass{2}}
┌ ┐
- <=50K ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.98
- >50K ┤■ 0.02
+ <=50K ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.91
+ >50K ┤■■■ 0.09
└ ┘
````
@@ -629,7 +632,7 @@ accuracy(ypoint, y[test])
````
````
-0.7822081179300814
+0.7823616727235502
````
````@julia
@@ -637,7 +640,7 @@ log_loss(yprob, y[test])
````
````
-1.5659651402227337
+1.5655836383744342
````
Evaluate with one command:
@@ -653,19 +656,20 @@ evaluate(
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: RandomForestClassifier-400
+Tag: RandomForestClassifier-308
Extract:
┌──────────────────────┬──────────────┬─────────────┐
│ measure │ operation │ measurement │
├──────────────────────┼──────────────┼─────────────┤
│ Accuracy() │ predict_mode │ 0.781 │
-│ LogLoss( │ predict │ 1.53 │
+│ LogLoss( │ predict │ 1.55 │
│ tol = 2.22045e-16) │ │ │
└──────────────────────┴──────────────┴─────────────┘
-
+Apply `describe` to this result for a named tuple summary.
````
---
diff --git a/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md b/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md
index 47968ab..54350fb 100644
--- a/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md
+++ b/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md
@@ -142,24 +142,25 @@ e1 = evaluate(pipe, X, y; resampling=CV(nfolds=4, rng=123), repeats=2, measure=m
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicPipeline-381
+Tag: DeterministicPipeline-639
Extract:
-┌──────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 180000.0 │
-│ p = 1) │ │ │
-└──────────┴───────────┴─────────────┘
+┌──────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼───────────┼─────────────┼─────────┤
+│ LPLoss( │ predict │ 179980.0 │ 630.0 │
+│ p = 1) │ │ │ │
+└──────────┴───────────┴─────────────┴─────────┘
┌───────────────────────────────────────────────────────────────────────────────
│ per_fold ⋯
├───────────────────────────────────────────────────────────────────────────────
│ [180000.0, 180000.0, 181000.0, 179000.0, 179000.0, 180000.0, 180000.0, 18100 ⋯
└───────────────────────────────────────────────────────────────────────────────
- 2 columns omitted
-
+ 1 column omitted
+Apply `describe` to this result for a named tuple summary.
````
Notice the target very large on the current scale:
@@ -206,24 +207,24 @@ e2 = evaluate(norm_pipe, X, y; resampling=CV(nfolds=4, rng=123), repeats=2, meas
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: TransformedTargetModelDeterministic-404
+Tag: TransformedTargetModelDeterministic-399
Extract:
-┌──────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 19800.0 │
-│ p = 1) │ │ │
-└──────────┴───────────┴─────────────┘
-┌──────────────────────────────────────────────────────────────────────────┬────
-│ per_fold │ 1 ⋯
-├──────────────────────────────────────────────────────────────────────────┼────
-│ [18100.0, 19800.0, 21400.0, 19600.0, 20300.0, 19500.0, 18700.0, 20900.0] │ 8 ⋯
-└──────────────────────────────────────────────────────────────────────────┴────
- 1 column omitted
-
+┌──────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼───────────┼─────────────┼─────────┤
+│ LPLoss( │ predict │ 19780.0 │ 800.0 │
+│ p = 1) │ │ │ │
+└──────────┴───────────┴─────────────┴─────────┘
+┌──────────────────────────────────────────────────────────────────────────┐
+│ per_fold │
+├──────────────────────────────────────────────────────────────────────────┤
+│ [18100.0, 19800.0, 21400.0, 19600.0, 20300.0, 19500.0, 18700.0, 20900.0] │
+└──────────────────────────────────────────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
Changing the regularization parameter `lambda` of ridge regressor, we can arrange that
@@ -248,9 +249,9 @@ evaluations = evaluate(
````
3-element Vector{PerformanceEvaluation{M, Vector{StatisticalMeasuresBase.RobustMeasure{StatisticalMeasuresBase.FussyMeasure{StatisticalMeasuresBase.RobustMeasure{StatisticalMeasuresBase.Multimeasure{StatisticalMeasuresBase.SupportsMissingsMeasure{StatisticalMeasures.LPLossOnScalars{Int64}}, Nothing, StatisticalMeasuresBase.Mean, typeof(identity)}}, Nothing}}}, Vector{Float64}, Vector{Float64}, Vector{typeof(predict)}, Vector{Vector{Float64}}, Vector{Vector{Vector{Float64}}}, FittedParamsPerFold, ReportPerFold, CV} where {M, FittedParamsPerFold, ReportPerFold}}:
- PerformanceEvaluation("default lambda", 180000.0 ± 635.0)
- PerformanceEvaluation("new lambda", 180000.0 ± 1470.0)
- PerformanceEvaluation("new lambda & normalized target", 18500.0 ± 443.0)
+ PerformanceEvaluation("default lambda", 179980.0 ± 630.0)
+ PerformanceEvaluation("new lambda", 180100.0 ± 1500.0)
+ PerformanceEvaluation("new lambda & normalized target", 18460.0 ± 440.0)
````
Here's a pretty view of these results:
diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg b/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg
index 55dc002..1687109 100644
--- a/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg
+++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg
@@ -1,46 +1,46 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md b/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md
index 13d89b2..286e160 100644
--- a/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md
+++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md
@@ -77,23 +77,24 @@ ebase = evaluate(baseline, X, y; options...)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: ProbabilisticPipeline-112
+Tag: ProbabilisticPipeline-908
Extract:
-┌──────────┬──────────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼──────────────┼─────────────┤
-│ LPLoss( │ predict_mean │ 0.568 │
-│ p = 1) │ │ │
-└──────────┴──────────────┴─────────────┘
-┌──────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├──────────────────────┼─────────┤
-│ [0.544, 0.57, 0.592] │ 0.0332 │
-└──────────────────────┴─────────┘
-
+┌──────────┬──────────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼──────────────┼─────────────┼─────────┤
+│ LPLoss( │ predict_mean │ 0.568 │ 0.033 │
+│ p = 1) │ │ │ │
+└──────────┴──────────────┴─────────────┴─────────┘
+┌──────────────────────┐
+│ per_fold │
+├──────────────────────┤
+│ [0.544, 0.57, 0.592] │
+└──────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
Inspect hyper-parameters:
@@ -147,7 +148,7 @@ savefig("learning_curve.svg");
````
[ Info: Training machine(DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …), …).
[ Info: Attempting to evaluate 30 models.
-
Evaluating over 30 metamodels: 7%[=> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 10%[==> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 13%[===> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 17%[====> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 20%[=====> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 23%[=====> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 27%[======> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 30%[=======> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 33%[========> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 37%[=========> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 40%[==========> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 43%[==========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 47%[===========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 50%[============> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 53%[=============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 57%[==============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 60%[===============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 63%[===============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 67%[================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 70%[=================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 73%[==================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 77%[===================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 80%[====================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 83%[====================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 87%[=====================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 90%[======================> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 93%[=======================> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 97%[========================>] ETA: 0:00:01[K
Evaluating over 30 metamodels: 100%[=========================] Time: 0:00:36[K
+
Evaluating over 30 metamodels: 7%[=> ] ETA: 0:00:00[K
Evaluating over 30 metamodels: 10%[==> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 13%[===> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 17%[====> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 20%[=====> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 23%[=====> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 27%[======> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 30%[=======> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 33%[========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 37%[=========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 40%[==========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 43%[==========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 47%[===========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 50%[============> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 53%[=============> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 57%[==============> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 60%[===============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 63%[===============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 67%[================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 70%[=================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 73%[==================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 77%[===================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 80%[====================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 83%[====================> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 87%[=====================> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 90%[======================> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 93%[=======================> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 97%[========================>] ETA: 0:00:01[K
Evaluating over 30 metamodels: 100%[=========================] Time: 0:00:28[K
````
@@ -224,7 +225,7 @@ savefig("tuned_model_grid.svg");
````
[ Info: Training machine(DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …), …).
[ Info: Attempting to evaluate 36 models.
-
Evaluating over 36 metamodels: 6%[=> ] ETA: 0:00:33[K
Evaluating over 36 metamodels: 8%[==> ] ETA: 0:00:45[K
Evaluating over 36 metamodels: 11%[==> ] ETA: 0:00:42[K
Evaluating over 36 metamodels: 14%[===> ] ETA: 0:00:37[K
Evaluating over 36 metamodels: 17%[====> ] ETA: 0:00:42[K
Evaluating over 36 metamodels: 19%[====> ] ETA: 0:00:39[K
Evaluating over 36 metamodels: 22%[=====> ] ETA: 0:00:36[K
Evaluating over 36 metamodels: 25%[======> ] ETA: 0:00:36[K
Evaluating over 36 metamodels: 28%[======> ] ETA: 0:00:34[K
Evaluating over 36 metamodels: 31%[=======> ] ETA: 0:00:33[K
Evaluating over 36 metamodels: 33%[========> ] ETA: 0:00:30[K
Evaluating over 36 metamodels: 36%[=========> ] ETA: 0:00:29[K
Evaluating over 36 metamodels: 39%[=========> ] ETA: 0:00:28[K
Evaluating over 36 metamodels: 42%[==========> ] ETA: 0:00:28[K
Evaluating over 36 metamodels: 44%[===========> ] ETA: 0:00:28[K
Evaluating over 36 metamodels: 47%[===========> ] ETA: 0:00:27[K
Evaluating over 36 metamodels: 50%[============> ] ETA: 0:00:26[K
Evaluating over 36 metamodels: 53%[=============> ] ETA: 0:00:25[K
Evaluating over 36 metamodels: 56%[=============> ] ETA: 0:00:24[K
Evaluating over 36 metamodels: 58%[==============> ] ETA: 0:00:22[K
Evaluating over 36 metamodels: 61%[===============> ] ETA: 0:00:21[K
Evaluating over 36 metamodels: 64%[===============> ] ETA: 0:00:19[K
Evaluating over 36 metamodels: 67%[================> ] ETA: 0:00:18[K
Evaluating over 36 metamodels: 69%[=================> ] ETA: 0:00:16[K
Evaluating over 36 metamodels: 72%[==================> ] ETA: 0:00:15[K
Evaluating over 36 metamodels: 75%[==================> ] ETA: 0:00:13[K
Evaluating over 36 metamodels: 78%[===================> ] ETA: 0:00:12[K
Evaluating over 36 metamodels: 81%[====================> ] ETA: 0:00:10[K
Evaluating over 36 metamodels: 83%[====================> ] ETA: 0:00:09[K
Evaluating over 36 metamodels: 86%[=====================> ] ETA: 0:00:08[K
Evaluating over 36 metamodels: 89%[======================> ] ETA: 0:00:06[K
Evaluating over 36 metamodels: 92%[======================> ] ETA: 0:00:04[K
Evaluating over 36 metamodels: 94%[=======================> ] ETA: 0:00:03[K
Evaluating over 36 metamodels: 97%[========================>] ETA: 0:00:01[K
Evaluating over 36 metamodels: 100%[=========================] Time: 0:00:53[K
+
Evaluating over 36 metamodels: 6%[=> ] ETA: 0:00:43[K
Evaluating over 36 metamodels: 8%[==> ] ETA: 0:00:50[K
Evaluating over 36 metamodels: 11%[==> ] ETA: 0:00:43[K
Evaluating over 36 metamodels: 14%[===> ] ETA: 0:00:44[K
Evaluating over 36 metamodels: 17%[====> ] ETA: 0:00:42[K
Evaluating over 36 metamodels: 19%[====> ] ETA: 0:00:40[K
Evaluating over 36 metamodels: 22%[=====> ] ETA: 0:00:39[K
Evaluating over 36 metamodels: 25%[======> ] ETA: 0:00:35[K
Evaluating over 36 metamodels: 28%[======> ] ETA: 0:00:34[K
Evaluating over 36 metamodels: 31%[=======> ] ETA: 0:00:34[K
Evaluating over 36 metamodels: 33%[========> ] ETA: 0:00:33[K
Evaluating over 36 metamodels: 36%[=========> ] ETA: 0:00:30[K
Evaluating over 36 metamodels: 39%[=========> ] ETA: 0:00:30[K
Evaluating over 36 metamodels: 42%[==========> ] ETA: 0:00:29[K
Evaluating over 36 metamodels: 44%[===========> ] ETA: 0:00:28[K
Evaluating over 36 metamodels: 47%[===========> ] ETA: 0:00:26[K
Evaluating over 36 metamodels: 50%[============> ] ETA: 0:00:25[K
Evaluating over 36 metamodels: 53%[=============> ] ETA: 0:00:23[K
Evaluating over 36 metamodels: 56%[=============> ] ETA: 0:00:22[K
Evaluating over 36 metamodels: 58%[==============> ] ETA: 0:00:20[K
Evaluating over 36 metamodels: 61%[===============> ] ETA: 0:00:19[K
Evaluating over 36 metamodels: 64%[===============> ] ETA: 0:00:17[K
Evaluating over 36 metamodels: 67%[================> ] ETA: 0:00:16[K
Evaluating over 36 metamodels: 69%[=================> ] ETA: 0:00:15[K
Evaluating over 36 metamodels: 72%[==================> ] ETA: 0:00:13[K
Evaluating over 36 metamodels: 75%[==================> ] ETA: 0:00:12[K
Evaluating over 36 metamodels: 78%[===================> ] ETA: 0:00:11[K
Evaluating over 36 metamodels: 81%[====================> ] ETA: 0:00:09[K
Evaluating over 36 metamodels: 83%[====================> ] ETA: 0:00:08[K
Evaluating over 36 metamodels: 86%[=====================> ] ETA: 0:00:07[K
Evaluating over 36 metamodels: 89%[======================> ] ETA: 0:00:05[K
Evaluating over 36 metamodels: 92%[======================> ] ETA: 0:00:04[K
Evaluating over 36 metamodels: 94%[=======================> ] ETA: 0:00:03[K
Evaluating over 36 metamodels: 97%[========================>] ETA: 0:00:01[K
Evaluating over 36 metamodels: 100%[=========================] Time: 0:00:48[K
````
@@ -271,18 +272,18 @@ CompactPerformanceEvaluation object with these fields:
resampling, repeats
Tag:
Extract:
-┌──────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 0.171 │
-│ p = 1) │ │ │
-└──────────┴───────────┴─────────────┘
-┌─────────────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├─────────────────────────────┼─────────┤
-│ [0.174, 0.16, 0.156, 0.192] │ 0.0181 │
-└─────────────────────────────┴─────────┘
-
+┌──────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼───────────┼─────────────┼─────────┤
+│ LPLoss( │ predict │ 0.171 │ 0.018 │
+│ p = 1) │ │ │ │
+└──────────┴───────────┴─────────────┴─────────┘
+┌─────────────────────────────┐
+│ per_fold │
+├─────────────────────────────┤
+│ [0.174, 0.16, 0.156, 0.192] │
+└─────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
Instead we can use a random search strategy:
@@ -306,7 +307,7 @@ savefig("tuned_model_random.svg");
````
[ Info: Training machine(DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …), …).
[ Info: Attempting to evaluate 40 models.
-
Evaluating over 40 metamodels: 0%[> ] ETA: N/A[K
Evaluating over 40 metamodels: 2%[> ] ETA: 0:00:32[K
Evaluating over 40 metamodels: 5%[=> ] ETA: 0:00:47[K
Evaluating over 40 metamodels: 8%[=> ] ETA: 0:00:44[K
Evaluating over 40 metamodels: 10%[==> ] ETA: 0:00:46[K
Evaluating over 40 metamodels: 12%[===> ] ETA: 0:00:53[K
Evaluating over 40 metamodels: 15%[===> ] ETA: 0:00:49[K
Evaluating over 40 metamodels: 18%[====> ] ETA: 0:00:47[K
Evaluating over 40 metamodels: 20%[=====> ] ETA: 0:00:46[K
Evaluating over 40 metamodels: 22%[=====> ] ETA: 0:00:47[K
Evaluating over 40 metamodels: 25%[======> ] ETA: 0:00:46[K
Evaluating over 40 metamodels: 28%[======> ] ETA: 0:00:43[K
Evaluating over 40 metamodels: 30%[=======> ] ETA: 0:00:42[K
Evaluating over 40 metamodels: 32%[========> ] ETA: 0:00:40[K
Evaluating over 40 metamodels: 35%[========> ] ETA: 0:00:40[K
Evaluating over 40 metamodels: 38%[=========> ] ETA: 0:00:39[K
Evaluating over 40 metamodels: 40%[==========> ] ETA: 0:00:37[K
Evaluating over 40 metamodels: 42%[==========> ] ETA: 0:00:36[K
Evaluating over 40 metamodels: 45%[===========> ] ETA: 0:00:34[K
Evaluating over 40 metamodels: 48%[===========> ] ETA: 0:00:33[K
Evaluating over 40 metamodels: 50%[============> ] ETA: 0:00:32[K
Evaluating over 40 metamodels: 52%[=============> ] ETA: 0:00:30[K
Evaluating over 40 metamodels: 55%[=============> ] ETA: 0:00:28[K
Evaluating over 40 metamodels: 58%[==============> ] ETA: 0:00:27[K
Evaluating over 40 metamodels: 60%[===============> ] ETA: 0:00:25[K
Evaluating over 40 metamodels: 62%[===============> ] ETA: 0:00:24[K
Evaluating over 40 metamodels: 65%[================> ] ETA: 0:00:22[K
Evaluating over 40 metamodels: 68%[================> ] ETA: 0:00:21[K
Evaluating over 40 metamodels: 70%[=================> ] ETA: 0:00:19[K
Evaluating over 40 metamodels: 72%[==================> ] ETA: 0:00:17[K
Evaluating over 40 metamodels: 75%[==================> ] ETA: 0:00:15[K
Evaluating over 40 metamodels: 78%[===================> ] ETA: 0:00:14[K
Evaluating over 40 metamodels: 80%[====================> ] ETA: 0:00:12[K
Evaluating over 40 metamodels: 82%[====================> ] ETA: 0:00:11[K
Evaluating over 40 metamodels: 85%[=====================> ] ETA: 0:00:09[K
Evaluating over 40 metamodels: 88%[=====================> ] ETA: 0:00:08[K
Evaluating over 40 metamodels: 90%[======================> ] ETA: 0:00:06[K
Evaluating over 40 metamodels: 92%[=======================> ] ETA: 0:00:05[K
Evaluating over 40 metamodels: 95%[=======================> ] ETA: 0:00:03[K
Evaluating over 40 metamodels: 98%[========================>] ETA: 0:00:02[K
Evaluating over 40 metamodels: 100%[=========================] Time: 0:01:01[K
+
Evaluating over 40 metamodels: 5%[=> ] ETA: 0:00:56[K
Evaluating over 40 metamodels: 8%[=> ] ETA: 0:00:48[K
Evaluating over 40 metamodels: 10%[==> ] ETA: 0:00:48[K
Evaluating over 40 metamodels: 12%[===> ] ETA: 0:00:48[K
Evaluating over 40 metamodels: 15%[===> ] ETA: 0:00:44[K
Evaluating over 40 metamodels: 18%[====> ] ETA: 0:00:46[K
Evaluating over 40 metamodels: 20%[=====> ] ETA: 0:00:44[K
Evaluating over 40 metamodels: 22%[=====> ] ETA: 0:00:42[K
Evaluating over 40 metamodels: 25%[======> ] ETA: 0:00:41[K
Evaluating over 40 metamodels: 28%[======> ] ETA: 0:00:38[K
Evaluating over 40 metamodels: 30%[=======> ] ETA: 0:00:39[K
Evaluating over 40 metamodels: 32%[========> ] ETA: 0:00:37[K
Evaluating over 40 metamodels: 35%[========> ] ETA: 0:00:36[K
Evaluating over 40 metamodels: 38%[=========> ] ETA: 0:00:35[K
Evaluating over 40 metamodels: 40%[==========> ] ETA: 0:00:33[K
Evaluating over 40 metamodels: 42%[==========> ] ETA: 0:00:33[K
Evaluating over 40 metamodels: 45%[===========> ] ETA: 0:00:31[K
Evaluating over 40 metamodels: 48%[===========> ] ETA: 0:00:29[K
Evaluating over 40 metamodels: 50%[============> ] ETA: 0:00:28[K
Evaluating over 40 metamodels: 52%[=============> ] ETA: 0:00:27[K
Evaluating over 40 metamodels: 55%[=============> ] ETA: 0:00:26[K
Evaluating over 40 metamodels: 58%[==============> ] ETA: 0:00:24[K
Evaluating over 40 metamodels: 60%[===============> ] ETA: 0:00:23[K
Evaluating over 40 metamodels: 62%[===============> ] ETA: 0:00:22[K
Evaluating over 40 metamodels: 65%[================> ] ETA: 0:00:20[K
Evaluating over 40 metamodels: 68%[================> ] ETA: 0:00:19[K
Evaluating over 40 metamodels: 70%[=================> ] ETA: 0:00:17[K
Evaluating over 40 metamodels: 72%[==================> ] ETA: 0:00:15[K
Evaluating over 40 metamodels: 75%[==================> ] ETA: 0:00:14[K
Evaluating over 40 metamodels: 78%[===================> ] ETA: 0:00:12[K
Evaluating over 40 metamodels: 80%[====================> ] ETA: 0:00:11[K
Evaluating over 40 metamodels: 82%[====================> ] ETA: 0:00:10[K
Evaluating over 40 metamodels: 85%[=====================> ] ETA: 0:00:08[K
Evaluating over 40 metamodels: 88%[=====================> ] ETA: 0:00:07[K
Evaluating over 40 metamodels: 90%[======================> ] ETA: 0:00:06[K
Evaluating over 40 metamodels: 92%[=======================> ] ETA: 0:00:04[K
Evaluating over 40 metamodels: 95%[=======================> ] ETA: 0:00:03[K
Evaluating over 40 metamodels: 98%[========================>] ETA: 0:00:01[K
Evaluating over 40 metamodels: 100%[=========================] Time: 0:00:55[K
````
@@ -336,23 +337,24 @@ e1 = evaluate(tuned_pipe, X, y; options...)
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicTunedModel-250
+Tag: DeterministicTunedModel-569
Extract:
-┌──────────┬───────────┬─────────────┐
-│ measure │ operation │ measurement │
-├──────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 0.175 │
-│ p = 1) │ │ │
-└──────────┴───────────┴─────────────┘
-┌───────────────────────┬─────────┐
-│ per_fold │ 1.96*SE │
-├───────────────────────┼─────────┤
-│ [0.166, 0.166, 0.192] │ 0.021 │
-└───────────────────────┴─────────┘
-
+┌──────────┬───────────┬─────────────┬─────────┐
+│ measure │ operation │ measurement │ 1.96*SE │
+├──────────┼───────────┼─────────────┼─────────┤
+│ LPLoss( │ predict │ 0.175 │ 0.021 │
+│ p = 1) │ │ │ │
+└──────────┴───────────┴─────────────┴─────────┘
+┌───────────────────────┐
+│ per_fold │
+├───────────────────────┤
+│ [0.166, 0.166, 0.192] │
+└───────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
Comparing with the baseline computed earlier:
@@ -362,9 +364,9 @@ Comparing with the baseline computed earlier:
````
````
-ebase = PerformanceEvaluation("ProbabilisticPipeline-112", 0.568 ± 0.0332)
-e0 = PerformanceEvaluation("DeterministicPipeline-658", 0.184 ± 0.0281)
-e1 = PerformanceEvaluation("DeterministicTunedModel-250", 0.175 ± 0.021)
+ebase = PerformanceEvaluation("ProbabilisticPipeline-908", 0.568 ± 0.033)
+e0 = PerformanceEvaluation("DeterministicPipeline-823", 0.184 ± 0.028)
+e1 = PerformanceEvaluation("DeterministicTunedModel-569", 0.175 ± 0.021)
````
@@ -380,8 +382,8 @@ describe.([e0, e1]) |> pretty
│ String │ Measurement{Float64} │
│ Textual │ Continuous │
├─────────────────────────────┼──────────────────────┤
-│ DeterministicPipeline-658 │ 0.184±0.028 │
-│ DeterministicTunedModel-250 │ 0.175±0.021 │
+│ DeterministicPipeline-823 │ 0.184±0.028 │
+│ DeterministicTunedModel-569 │ 0.175±0.021 │
└─────────────────────────────┴──────────────────────┘
````
diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg
index c80b3f8..94ad1ac 100644
--- a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg
+++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg
@@ -1,294 +1,294 @@
diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg
index 9911fd3..33a2310 100644
--- a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg
+++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg
@@ -1,269 +1,269 @@
diff --git a/docs/src/notebooks/lightning_tour/notebook.md b/docs/src/notebooks/lightning_tour/notebook.md
index ba6449b..c912761 100644
--- a/docs/src/notebooks/lightning_tour/notebook.md
+++ b/docs/src/notebooks/lightning_tour/notebook.md
@@ -246,8 +246,8 @@ mach = machine(self_tuning_pipe, X, y)
untrained Machine; does not cache data
model: DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …)
args:
- 1: Source @477 ⏎ ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}}
- 2: Source @252 ⏎ AbstractVector{ScientificTypesBase.Continuous}
+ 1: Source @634 ⏎ ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}}
+ 2: Source @471 ⏎ AbstractVector{ScientificTypesBase.Continuous}
````
@@ -261,9 +261,9 @@ first(yhat, 3)
````
3-element Vector{Float32}:
- -0.42176783
- 0.82785225
- 0.09056717
+ -2.8776388
+ -0.45805833
+ -0.8760577
````
Evaluating the "self-tuning" pipeline model's performance using all data and 5-fold
@@ -281,25 +281,26 @@ evaluate!(
````
PerformanceEvaluation object with these fields:
model, tag, measure, operation,
- measurement, uncertainty_radius_95, per_fold, per_observation,
+ measurement (per-fold aggregate), uncertainty_radius_95 (1.96*SE),
+ per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicTunedModel-354
+Tag: DeterministicTunedModel-157
Extract:
-┌───┬────────────┬───────────┬─────────────┐
-│ │ measure │ operation │ measurement │
-├───┼────────────┼───────────┼─────────────┤
-│ A │ LPLoss( │ predict │ 0.309 │
-│ │ p = 1) │ │ │
-│ B │ RSquared() │ predict │ 0.946 │
-└───┴────────────┴───────────┴─────────────┘
-┌───┬─────────────────────────────────────┬─────────┐
-│ │ per_fold │ 1.96*SE │
-├───┼─────────────────────────────────────┼─────────┤
-│ A │ [0.211, 0.279, 0.414, 0.334, 0.305] │ 0.0729 │
-│ B │ [0.967, 0.957, 0.902, 0.944, 0.962] │ 0.0258 │
-└───┴─────────────────────────────────────┴─────────┘
-
+┌───┬────────────┬───────────┬─────────────┬─────────┐
+│ │ measure │ operation │ measurement │ 1.96*SE │
+├───┼────────────┼───────────┼─────────────┼─────────┤
+│ A │ LPLoss( │ predict │ 0.17 │ 0.023 │
+│ │ p = 1) │ │ │ │
+│ B │ RSquared() │ predict │ 0.9699 │ 0.009 │
+└───┴────────────┴───────────┴─────────────┴─────────┘
+┌───┬────────────────────────────────────┐
+│ │ per_fold │
+├───┼────────────────────────────────────┤
+│ A │ [0.155, 0.159, 0.2, 0.19, 0.146] │
+│ B │ [0.96, 0.981, 0.962, 0.969, 0.977] │
+└───┴────────────────────────────────────┘
+Apply `describe` to this result for a named tuple summary.
````
Compare to a dummy model:
@@ -319,7 +320,7 @@ describe.(evaluations) |> pretty
````
[ Info: Performing evaluations using 1 thread.
-
Evaluating over 5 folds: 40%[==========> ] ETA: 0:00:27[K
Evaluating over 5 folds: 60%[===============> ] ETA: 0:00:15[K
Evaluating over 5 folds: 80%[====================> ] ETA: 0:00:07[K
Evaluating over 5 folds: 100%[=========================] Time: 0:00:33[K
+
Evaluating over 5 folds: 40%[==========> ] ETA: 0:00:11[K
Evaluating over 5 folds: 60%[===============> ] ETA: 0:00:07[K
Evaluating over 5 folds: 80%[====================> ] ETA: 0:00:04[K
Evaluating over 5 folds: 100%[=========================] Time: 0:00:18[K
[ Info: Performing evaluations using 1 thread.
Evaluating over 5 folds: 40%[==========> ] ETA: 0:00:01[K
Evaluating over 5 folds: 100%[=========================] Time: 0:00:00[K
┌─────────┬──────────────────────┬──────────────────────┐
@@ -327,8 +328,8 @@ describe.(evaluations) |> pretty
│ String │ Measurement{Float64} │ Measurement{Float64} │
│ Textual │ Continuous │ Continuous │
├─────────┼──────────────────────┼──────────────────────┤
-│ booster │ 0.309±0.073 │ 0.946±0.026 │
-│ dummy │ 1.48±0.15 │ -0.014±0.021 │
+│ booster │ 0.17±0.023 │ 0.9699±0.009 │
+│ dummy │ 1.13±0.15 │ -0.0044±0.0051 │
└─────────┴──────────────────────┴──────────────────────┘
````
diff --git a/learning_curve.svg b/learning_curve.svg
index dd9bd14..42b059d 100644
--- a/learning_curve.svg
+++ b/learning_curve.svg
@@ -1,46 +1,46 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/tuned_model_grid.svg b/tuned_model_grid.svg
index 358a08e..e3ed916 100644
--- a/tuned_model_grid.svg
+++ b/tuned_model_grid.svg
@@ -1,294 +1,294 @@
diff --git a/tuned_model_random.svg b/tuned_model_random.svg
index f06d6fe..7f4925e 100644
--- a/tuned_model_random.svg
+++ b/tuned_model_random.svg
@@ -1,269 +1,269 @@