Skip to content

Commit f6a4f1e

Browse files
Initial commit. Still WIP
1 parent 3b9c372 commit f6a4f1e

File tree

1 file changed

+31
-1
lines changed

1 file changed

+31
-1
lines changed

docs/core/whats-new/dotnet-9/overview.md

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,37 @@ The .NET 9 SDK introduces _workload sets_, where all of your workloads stay at a
5353

5454
For more information, see [What's new in the SDK for .NET 9](sdk.md).
5555

56-
## AI and Machine Learning
56+
## AI Building Blocks and Fundamentals
57+
58+
### Tokenizers
59+
60+
The Microsoft.ML.Tokenizers library provides .NET developer with capabilities for encoding and decoding text to tokens. For AI scenarios, this is important to manage context, calculate cost, and pre-process text when working with local models.
61+
62+
The latest release introduces significant new capabilities:
63+
64+
- Tokenizers
65+
- SentencePiece
66+
- WordPiece
67+
- BERT
68+
- CodeGen
69+
- Built-in tokenizers for the following models:
70+
- GPT (3, 3.5, 4, 4o, o1)
71+
- Llama
72+
- Phi
73+
74+
### Tensors
75+
76+
<!-- TODO -->
77+
78+
### Microsoft.Extensions.AI
79+
80+
<!-- TODO -->
81+
82+
### Microsoft.Extensions.VectorData
83+
84+
<!-- TODO -->
85+
86+
## ML.NET
5787

5888
ML.NET is an open-source, cross-platform framework that enables integration of custom machine-learning models into .NET applications. The latest version, ML.NET 4.0, adds [additional tokenizer support](../../../machine-learning/whats-new/overview.md#additional-tokenizer-support) for tokenizers such as Tiktoken and models such as Llama and CodeGen. <!--Add info about `Tensor<T>` here and in what's new for ML.NET.-->
5989

0 commit comments

Comments
 (0)