Skip to content

Commit 517100e

Browse files
new post: tut
1 parent 0271b59 commit 517100e

File tree

11 files changed

+5717
-0
lines changed

11 files changed

+5717
-0
lines changed

index.html

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,23 @@ <h1>
2424
Hello and welcome to my blog! As a postdoctoral researcher in machine learning and artificial intelligence, I’m here to share my journey, insights, and notes with you. I strive to be as clear and grounded as possible, and I hope you’ll find something interesting and valuable in my posts.
2525
</p>
2626

27+
<div onclick="location.href='posts/06-03-25-tut/index.html';" style="cursor: pointer;">
28+
<article class="light-post-theme post-block">
29+
<div class="post-inner">
30+
<div class="post-title">
31+
<img src="icons/pin.svg" width=16px height=16px/> Mid-Training Untying: A Fix That Barely Fixes Anything
32+
</div>
33+
<div class="post-desc">
34+
In one of the previous posts, we have seen how tying embeddings can be destabilize the training if the the data do not satisfy certain assumptions (see <a href="https://openreview.net/forum?id=yyYMAprcAR">here</a>). In this post, we will explore a simple idea to get the best of both worlds: early training boost with tied embeddings and late training stability with untied one. This was a research idea that I had in mind however it did not work as well as expected so I decided to share it here.
35+
</div>
36+
<div class="post-foot">
37+
<img src="icons/clock.svg" width=12px height=12px/> Date: 06 March, 2025 | Estimated Reading Time: ~5 min
38+
</div>
39+
</div>
40+
</article>
41+
</div>
42+
<br>
43+
2744
<div onclick="location.href='posts/29-12-24-chaos-II/index.html';" style="cursor: pointer;">
2845
<article class="light-post-theme post-block">
2946
<div class="post-inner">

posts/06-03-25-tut/acc-zoomed.pdf

-25 KB
Binary file not shown.

posts/06-03-25-tut/acc-zoomed.png

41.6 KB
Loading

0 commit comments

Comments
 (0)