File tree Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Original file line number Diff line number Diff line change @@ -92,7 +92,7 @@ class PeftInputAutocastDisableHook(ModelHook):
9292            1. Making forward implementations independent of device/dtype casting operations as much as possible. 
9393            2. Peforming inference without losing information from casting to different precisions. With the current 
9494               PEFT implementation (as linked in the reference above), and assuming running layerwise casting inference 
95-                with storage_dtype=torch.float8_e4m3fn and compute_dtype=torch.bfloat16, inputs are casted  to 
95+                with storage_dtype=torch.float8_e4m3fn and compute_dtype=torch.bfloat16, inputs are cast  to 
9696               torch.float8_e4m3fn in the lora layer. We will then upcast back to torch.bfloat16 when we continue the 
9797               forward pass in PEFT linear forward or Diffusers layer forward, with a `send_to_dtype` operation from 
9898               LayerwiseCastingHook. This will be a lossy operation and result in poorer generation quality. 
    
 
   
 
     
   
   
          
     
  
    
     
 
    
      
     
 
     
    You can’t perform that action at this time.
  
 
    
  
     
    
      
        
     
 
       
      
     
   
 
    
    
  
 
  
 
     
    
0 commit comments