Skip to content

@bluorion-com bluorion.com

Change the repository type filter

All

    Repositories list

    19 repositories

    • ZClip

      Public
      Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".
      lossgradient-clippingadaptive-gradient-clippingloss-spikeadaptive-clippingzclipstable-llm-pretrainingenable-high-learning-ratetraning-stabilitypre-training-stability
      Python
      •
      Apache License 2.0
      •10•145•3•3•Updated Nov 20, 2025Nov 20, 2025

    • perftest

      Public
      Shell
      •0•0•0•0•Updated Nov 3, 2025Nov 3, 2025

    • weight_rescaling

      Public
      Official implementation of the "Variance control via weight rescaling in LLM pretraining" paper.
      Python
      •
      Apache License 2.0
      •0•5•0•0•Updated Jun 29, 2025Jun 29, 2025

    • refine_massive_activations

      Public
      Official implementation of the paper: "A Refined Analysis of Massive Activations in LLMs".
      Python
      •
      MIT License
      •3•11•0•0•Updated May 21, 2025May 21, 2025

    • raydp

      Public
      RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.
      Scala
      •
      Apache License 2.0
      •79•0•0•0•Updated Apr 22, 2025Apr 22, 2025

    • Megatron-LM

      Public
      Ongoing research training transformer models at scale
      Python
      •
      Other
      •3.6k•1•0•0•Updated Mar 21, 2025Mar 21, 2025

    • vllm-production-stack

      Public
      vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization
      Python
      •
      Apache License 2.0
      •370•0•0•0•Updated Mar 14, 2025Mar 14, 2025

    • lingua

      Public
      Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •268•0•0•0•Updated Feb 6, 2025Feb 6, 2025

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •2.4k•0•0•0•Updated Jan 16, 2025Jan 16, 2025

    • torchx_nccl_test

      Public
      Python
      •0•0•0•0•Updated Jan 15, 2025Jan 15, 2025

    • blt

      Public
      Code for BLT research paper
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •189•0•0•0•Updated Jan 10, 2025Jan 10, 2025

    • check_nccl

      Public
      Python
      •0•0•0•0•Updated Nov 1, 2024Nov 1, 2024

    • netifaces

      Public
      C
      •
      MIT License
      •89•0•0•0•Updated Oct 7, 2024Oct 7, 2024

    • AdEMAMix-Optimizer-Pytorch

      Public
      The AdEMAMix Optimizer: Better, Faster, Older.
      Python
      •
      MIT License
      •10•0•0•0•Updated Oct 7, 2024Oct 7, 2024

    • wheels

      Public
      0•0•0•0•Updated Oct 4, 2024Oct 4, 2024

    • beam

      Public
      Apache Beam is a unified programming model for Batch and Streaming data processing.
      Java
      •
      Apache License 2.0
      •4.5k•0•0•0•Updated Oct 4, 2024Oct 4, 2024

    • lighteval

      Public
      LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and…
      Python
      •
      MIT License
      •429•0•0•0•Updated Aug 5, 2024Aug 5, 2024

    • rules_python

      Public
      Bazel Python Rules
      Starlark
      •
      Apache License 2.0
      •672•0•0•0•Updated Jul 31, 2024Jul 31, 2024

    • flash-attention-wheels

      Public
      C++
      •2•0•0•0•Updated Jul 29, 2024Jul 29, 2024