diff --git a/source/_posts/tensorflow.md b/source/_posts/tensorflow.md new file mode 100644 index 00000000..174a9009 --- /dev/null +++ b/source/_posts/tensorflow.md @@ -0,0 +1,170 @@ +--- +title: Tensorflow +date: 2025-11-04 +background: bg-[#ffa800] +tags: + - AI + - Python +categories: + - Other +intro: This is a quick reference list of cheat sheets for Tensorflow. See also [Tensorflow website](https://tensorflow.org/). +--- + +## Imports { .cols-1 } + +### General + +``` +import tensorflow as tf # root package +import tensorflow_datasets as tfds # dataset representation and loading +model.compile(optimizer, loss, metrics) # compile necessary components for training and evaluation +model.fit(x_train, y_train, epoch, batch_size) # model training +model.evaluate(x_test, y_test) # model evaluation +``` + +## Tensors { .cols-1 } + +### Basic Operations +``` +a = tf.constant(5) + tf.constant(3) # tf.constant is an immutable tensor storing the fixed value +a.numpy() # This will return the value, which is 8 +b = tf.Variable(10) # tf.Variable is a shared state for an entire execution time +b.assign(15) # this assign the new value to the variable +with tf.GradientTape() as tape: # record operations on variables for automatic differentiation +``` + +### Creation + +``` +x = +tf.random_normal_initializer(mean, std) # tensor with independent N(mean,stf) entries +tf.random_uniform_initializer(min_val, max_val) # tensor with independent Uniform(min_val, max_val) entries +x = tf.[ones|zeros](*size) # tensor with all 1's [or 0's] +y = x.clone() # clone of x +with torch.no_grad(): # code wrap that stops autograd from tracking tensor history +requires_grad=True # arg, when set to True, tracks computation + # history for future derivative calculations +``` + +### Dimensionality + +``` +tf.shape # shape of the tensor +tf.rank # number of dimension of the tensors +tf.size # number of elements in the tensor? +x = tf.concat(tensor_seq, axis=0) # concatenates tensors along axis +y = tf.reshape(tensor, [new_shape]) # reshapes x into size (a,b,...) +y = tf.reshape(tensor, [(-1,a]) # reshapes x into size (b,a) for some b +y = x.permute(*dims) # permutes dimensions +y = tf.expand_dims(x) # tensor with added axis +y = tf.expand_dims(x, axis=2) # (a,b,c) tensor -> (a,b,1,c) tensor +``` + +### Algebra + +``` +tf.add(a, b), a + b # matrix addition +tf.multiply(a, b), a * b # matrix-vector multiplication +tf.matmul(a, b), a @ b # matrix multiplication +tf.transpose() # matrix transpose +``` + +### GPU Usage + +``` +gpus = tf.config.list_physical_devices('GPU') # check whether there is a GPU usage +if gpus: + +tf.device() # manual device placement + # either "/CPU:0", "/GPU:0", or other qualified name + # of the second GPU of your machine + +try: + tf.config.set_visible_devices(gpus[0], 'GPU') # Limiting GPU memory growth + + +``` + +## Deep Learning Models { .cols-1 } + +### Creating Models +``` +tf.keras.Sequential # stack layers in a way that the computation + # will be performed sequentially +``` + +### Layers + +``` +tf.keras.layers.Dense(m,n) # fully connected layer from + # m to n units + +tf.keras.layers.ConvXd(m,n,s) # X dimensional conv layer from + # m to n channels where X⍷{1,2,3} + # and the kernel size is s + +tf.keras.layers.MaxPoolXd(s) # X dimension pooling layer + # (notation as above) + +tf.keras.layers.BatchNormalization # batch norm layer +tf.keras.layers.RNN/LSTM/GRU # recurrent layers +tf.keras.layers.Dropout(rate=0.5) # dropout layer for any dimensional input +tf.keras.layers.Embedding(input_dim, output_dim) # (tensor-wise) mapping from + # indices to embedding vectors +``` + +### Loss Functions + +``` +tf.keras.losses.X # where X is BinaryCrossentropy, BinaryFocalCrossentropy, CTC + # CategoricalCrossentropy, CategoricalFocalCrossentropy, + # CategoricalHinge, CosineSimilarity, Dice, Hinge, Huber + # KLDivergence, LogCosh, MeanAbsoluteError, MeanAbsolutePercentageError + # MeanSquaredError, MeanSquaredLogarithmicError, Poisson + # Reduction, SparseCategoricalCrossentropy, SquaredHinge, Tversky +``` + +### Activation Functions + +``` +tf.keras.activations.X # where X is ReLU, ReLU6, ELU, SELU, PReLU, LeakyReLU, + # RReLu, CELU, GELU, Threshold, Hardshrink, HardTanh, + # Sigmoid, LogSigmoid, Softplus, SoftShrink, + # Softsign, Tanh, TanhShrink, Softmin, Softmax, + # Softmax2d, LogSoftmax or AdaptiveSoftmaxWithLoss +``` + +### Optimizers + +``` +opt = tf.keras.optimizer.x(model.parameters(), ...) # create optimizer +opt.step() # update weights +optim.X # where X is SGD, Adadelta, Adafactor, + # Adagrad, Adam, AdamW, Adamax, Ftrl, Lion, + # LossScaleOptimizer ,RMSprop or Rprop +``` + +### Learning rate scheduling - Callbacks + +``` +callbacks = tf.keras.callbacks.LearningRateScheduler(scheduler) # create lr scheduler +model.fit(..., callbacks=[callback], ....) # update lr after optimizer updates weights + # using with fit(), evaluate(), and predict() +``` + +### Saving and Loading Models +``` +tf.keras.models.clone_model(...) # Clone a Functional or Sequential Model instance. +tf.keras.models.load_model(...) # Loads a model saved via model.save(). +tf.keras.models.model_from_json(...) # Parses a JSON model configuration string and returns a model instance. +tf.keras.models.save_model(...) # Saves a model as a .keras file. +``` + +## Data Utilities { .cols-1 } + +### Datasets + +``` +pip install tensorflow-datasets # install the module +tfds.load('mnist', split, shuffle_files) # loading a dataset +```