Implementation of ARC-AGI dataset loader (#3115)

* Implementation of linear_ layer for neural networks. This layer provides an optimized linear transformation for multi-dimensional inputs.

* Minor change

* Update dlib/dnn/layers.h

Co-authored-by: Copilot <[email protected]>

* Add reshape_to and flatten layers to Dlib's DNN module

* Missing update to "visitors.h"

* format fixing for reshape_to

* Update dlib/test/dnn.cpp

* Vocabulary size fixed for learning, and function added for transformation-free tokenization

* Added a new example for learning a “complex” Transformer model.

* Added a new example for learning a “complex” Transformer model.

* Updated example for training a Transformer model.

* fix for gcc/ffmpeg compilation

* Fix a warning message for Ubuntu compilation.

* Update for Linux environment.

* Fix batch building

* Slight improvement in model definition.

* linear_ layer implementation improvement

* finalizing the example

* Fixing break condition in training method.

* Fixing declaration order of variables.

* bpe_tokenizer improvements.

* Example updated.

* bpe_tokenizer class refactoring.

* Example updated.

* bpe_tokenizer class updated.

* Decoding part of the bpe_tokenizer updated.

* Network definition update

* Add Adaptive Computation Time (ACT) layer with CPU/CUDA support

* Fixes

* Update comments for params

* Fixes and improvements

* Disabling enable_depth_scaling, which obviously affects the result of test_layer

* Implementation of ARC-AGI dataset loader

---------

Co-authored-by: Davis E. King <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Aldric PIERRAIN <[email protected]>

Author: 4 people

dlib/data_io.h

@@ -7,6 +7,7 @@
 #include "data_io/image_dataset_metadata.h"
 #include "data_io/mnist.h"
 #include "data_io/cifar.h"
+#include "data_io/arc_agi.h"
 
 #ifndef DLIB_ISO_CPP_ONLY
 #include "data_io/load_image_dataset.h"