Merge pull request #73334 from Snowy1803/unified-lost-vars-statistics

Add compile flag for lost debug variables statistics

Author: Snowy1803

docs/DebuggingTheCompiler.md

@@ -302,6 +302,13 @@ with the proper attributes to ensure they'll be available in the debugger. In
 particular, if you see `SWIFT_DEBUG_DUMP` in a class declaration, that class
 has a `dump()` method you can call.
 
+### Pass statistics
+
+There are options to output a lot of different statistics, including about
+SIL passes. More information is available in
+[Compiler Performance](CompilerPerformance.md) for the unified statistics, and
+[Optimizer Counter Analysis](OptimizerCountersAnalysis.md) for pass counters.
+
 ## Debugging and Profiling on SIL level
 
 ### SIL source level profiling

docs/HowToUpdateDebugInfo.md

@@ -255,11 +255,18 @@ debug_value %1 : $Int, var, name "pair", type $Pair, expr op_fragment:#Pair.a //
 
 ## Rules of thumb
 - Optimization passes may never drop a variable entirely. If a variable is
-  entirely optimized away, an `undef` debug value should still be kept.
+  entirely optimized away, an `undef` debug value should still be kept. The only
+  exception is an unreachable function or scope, which is entirely removed.
 - A `debug_value` must always describe a correct value for that source variable
   at that source location. If a value is only correct on some paths through that
   instruction, it must be replaced with `undef`. Debug info never speculates.
 - When a SIL instruction is deleted, call salvageDebugInfo(). It will try to
   capture the effect of the deleted instruction in a debug expression, so the
   location can be preserved. You can also use an `InstructionDeleter` which will
   automatically call `salvageDebugInfo`.
+
+> [!Tip]
+> To detect when a pass drops a variable, you can use the
+> `-Xllvm -sil-stats-lost-variables` to print when a variable is lost by a pass.
+> More information about this option is available in
+> [Optimizer Counter Analysis](OptimizerCountersAnalysis.md)

docs/OptimizerCountersAnalysis.md

@@ -54,7 +54,8 @@ The following statistics can be recorded:
 
   * For SILFunctions: the number of SIL basic blocks for each SILFunction, the
     number of SIL instructions, the number of SILInstructions of a specific
-    kind (e.g. a number of alloc_ref instructions)
+    kind (e.g. a number of alloc_ref instructions), the number of debug
+    variables
 
   * For SILModules: the number of SIL basic blocks in the SILModule, the number
     of SIL instructions, the number of SILFunctions, the number of
@@ -118,6 +119,16 @@ e.g. `-Xllvm -sil-stats-only-instructions=alloc_ref,alloc_stack`. If you need to
 collect stats about all kinds of SIL instructions, you can use this syntax: 
 `-Xllvm -sil-stats-only-instructions=all`.
 
+### Debug variable level counters
+A different type of counter is the lost debug variables counter. It is enabled
+by using the `-Xllvm -sil-stats-lost-variables` command-line option. It only
+tracks statistics about lost variables in SILFunctions. It is not enabled by
+any other command-line option, but can be combined with the others. It is not
+compatible with thresholds, it always counts lost variables. Note that it does
+not track the number of debug variables: it counts the number of debug variables
+that were present, but aren't anymore. If a variable changes location or scope,
+which is not allowed, it will be counted as lost.
+
 ## Configuring which counters changes should be recorded
 
 The user has a possibility to configure a number of thresholds, which control
@@ -181,9 +192,9 @@ And for counter stats it looks like this:
   * `function_history` corresponds to the verbose mode of function
     counters collection, when changes to the SILFunction counters are logged 
     unconditionally, without any on-line filtering.
-* `CounterName` is typically one of `block`, `inst`, `function`, `memory`, 
-   or `inst_instruction_name` if you collect counters for specific kinds of SIL
-   instructions.
+* `CounterName` is typically one of `block`, `inst`, `function`, `memory`,
+   `lostvars`, or `inst_instruction_name` if you collect counters for specific
+   kinds of SIL instructions.
 * `Symbol` is e.g. the name of a function
 * `StageName` is the name of the current optimizer pipeline stage
 * `TransformName` is the name of the current optimizer transformation/pass
@@ -192,6 +203,14 @@ And for counter stats it looks like this:
    want to reproduce the result later using 
    `-Xllvm -sil-opt-pass-count -Xllvm TransformPassNumber`
 
+## Extract Lost Variables per Pass
+
+For lost variables, there is a script to output a CSV with only the amount of
+lost variables per pass. You can then easily open the resulting CSV in Numbers
+to make graphs.
+
+`utils/process-stats-lost-variables csv_file_with_counters > csv_aggregate`
+
 ## Storing the produced statistics into a database
 
 To store the set of produced counters into a database, you can use the
@@ -345,4 +364,3 @@ from Counters C where C.counter = 'inst' and C.kind = 'module'
 group by Stage
 order by sum(C.Delta);
 ```
-

lib/SILOptimizer/Utils/OptimizerStatsUtils.cpp

@@ -68,7 +68,8 @@
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Process.h"
-#include "swift/SIL/SILValue.h"
+#include "swift/Basic/SourceManager.h"
+#include "swift/SIL/DebugUtils.h"
 #include "swift/SIL/SILVisitor.h"
 #include "swift/SILOptimizer/Analysis/Analysis.h"
 #include "swift/SILOptimizer/PassManager/PassManager.h"
@@ -186,6 +187,11 @@ llvm::cl::opt<bool> SILStatsFunctions(
     "sil-stats-functions", llvm::cl::init(false),
     llvm::cl::desc("Enable computation of statistics for SIL functions"));
 
+/// Dump statistics about lost debug variables.
+llvm::cl::opt<bool> SILStatsLostVariables(
+    "sil-stats-lost-variables", llvm::cl::init(false),
+    llvm::cl::desc("Dump lost debug variables stats"));
+
 /// The name of the output file for optimizer counters.
 llvm::cl::opt<std::string> SILStatsOutputFile(
     "sil-stats-output-file", llvm::cl::init(""),
@@ -272,16 +278,28 @@ struct FunctionStat {
   /// Instruction counts per SILInstruction kind.
   InstructionCounts InstCounts;
 
+  using VarID = std::tuple<const SILDebugScope *, llvm::StringRef,
+                           unsigned, unsigned>;
+  llvm::StringSet<> VarNames;
+  llvm::DenseSet<FunctionStat::VarID> DebugVariables;
+
   FunctionStat(SILFunction *F);
   FunctionStat() {}
 
+  // The DebugVariables set contains pointers to VarNames. Disallow copy.
+  FunctionStat(const FunctionStat &) = delete;
+  FunctionStat(FunctionStat &&) = default;
+  FunctionStat &operator=(const FunctionStat &) = delete;
+  FunctionStat &operator=(FunctionStat &&) = default;
+
   void print(llvm::raw_ostream &stream) const {
     stream << "FunctionStat("
            << "blocks = " << BlockCount << ", Inst = " << InstCount << ")\n";
   }
 
   bool operator==(const FunctionStat &rhs) const {
-    return BlockCount == rhs.BlockCount && InstCount == rhs.InstCount;
+    return BlockCount == rhs.BlockCount && InstCount == rhs.InstCount
+      && DebugVariables == rhs.DebugVariables;
   }
 
   bool operator!=(const FunctionStat &rhs) const { return !(*this == rhs); }
@@ -360,14 +378,20 @@ struct ModuleStat {
   bool operator!=(const ModuleStat &rhs) const { return !(*this == rhs); }
 };
 
-// A helper type to collect the stats about the number of instructions and basic
-// blocks.
+/// A helper type to collect the stats about a function (instructions, blocks,
+/// debug variables).
 struct InstCountVisitor : SILInstructionVisitor<InstCountVisitor> {
   int BlockCount = 0;
   int InstCount = 0;
   InstructionCounts &InstCounts;
 
-  InstCountVisitor(InstructionCounts &InstCounts) : InstCounts(InstCounts) {}
+  llvm::StringSet<> &VarNames;
+  llvm::DenseSet<FunctionStat::VarID> &DebugVariables;
+
+  InstCountVisitor(InstructionCounts &InstCounts,
+                   llvm::StringSet<> &VarNames,
+                   llvm::DenseSet<FunctionStat::VarID> &DebugVariables)
+  : InstCounts(InstCounts), VarNames(VarNames), DebugVariables(DebugVariables) {}
 
   int getBlockCount() const {
     return BlockCount;
@@ -385,6 +409,29 @@ struct InstCountVisitor : SILInstructionVisitor<InstCountVisitor> {
   void visit(SILInstruction *I) {
     ++InstCount;
     ++InstCounts[I->getKind()];
+
+    if (!SILStatsLostVariables)
+      return;
+    // Check the debug variable.
+    DebugVarCarryingInst inst(I);
+    if (!inst)
+      return;
+    std::optional<SILDebugVariable> varInfo = inst.getVarInfo();
+    if (!varInfo)
+      return;
+
+    llvm::StringRef UniqueName = VarNames.insert(varInfo->Name).first->getKey();
+    if (!varInfo->Loc)
+      varInfo->Loc = inst->getLoc();
+    unsigned line = 0, col = 0;
+    if (varInfo->Loc && varInfo->Loc->getSourceLoc().isValid()) {
+      std::tie(line, col) = inst->getModule().getSourceManager()
+        .getPresumedLineAndColumnForLoc(varInfo->Loc->getSourceLoc(), 0);
+    }
+    FunctionStat::VarID key(
+              varInfo->Scope ? varInfo->Scope : inst->getDebugScope(),
+              UniqueName, line, col);
+    DebugVariables.insert(key);
   }
 };
 
@@ -549,7 +596,7 @@ class NewLineInserter {
 std::unique_ptr<llvm::raw_ostream, void(*)(llvm::raw_ostream *)>
     stats_output_stream = {nullptr, nullptr};
 
-/// Return the output streamm to be used for logging the collected statistics.
+/// Return the output stream to be used for logging the collected statistics.
 llvm::raw_ostream &stats_os() {
   // Initialize the stream if it is not initialized yet.
   if (!stats_output_stream) {
@@ -664,6 +711,18 @@ bool isFirstTimeData(int Old, int New) {
   return Old == 0 && New != Old;
 }
 
+int computeLostVariables(llvm::DenseSet<FunctionStat::VarID> &Old,
+                         llvm::DenseSet<FunctionStat::VarID> &New) {
+  unsigned count = 0;
+  for (auto &var : Old) {
+    if (New.contains(var))
+      continue;
+    // llvm::dbgs() << "Lost variable: " << std::get<1>(var) << "\n";
+    count++;
+  }
+  return count;
+}
+
 /// Dump statistics for a SILFunction. It is only used if a user asked to
 /// produce detailed stats about transformations of SILFunctions. This
 /// information is dumped unconditionally, for each transformation that changed
@@ -710,7 +769,7 @@ void processFuncStatsChanges(SILFunction *F, FunctionStat &OldStat,
                              TransformationContext &Ctx) {
   processFuncStatHistory(F, NewStat, Ctx);
 
-  if (!SILStatsFunctions && !SILStatsDumpAll)
+  if (!SILStatsFunctions && !SILStatsLostVariables && !SILStatsDumpAll)
     return;
 
   if (OldStat == NewStat)
@@ -724,6 +783,8 @@ void processFuncStatsChanges(SILFunction *F, FunctionStat &OldStat,
   // Compute deltas.
   double DeltaBlockCount = computeDelta(OldStat.BlockCount, NewStat.BlockCount);
   double DeltaInstCount = computeDelta(OldStat.InstCount, NewStat.InstCount);
+  int LostVariables = computeLostVariables(OldStat.DebugVariables,
+                                           NewStat.DebugVariables);
 
   NewLineInserter nl;
 
@@ -744,6 +805,11 @@ void processFuncStatsChanges(SILFunction *F, FunctionStat &OldStat,
     printCounterChange("function", "inst", DeltaInstCount, OldStat.InstCount,
                        NewStat.InstCount, Ctx, F->getName());
   }
+
+  if ((SILStatsDumpAll || SILStatsLostVariables) && LostVariables) {
+    stats_os() << nl.get();
+    printCounterValue("function", "lostvars", LostVariables, F->getName(), Ctx);
+  }
 }
 
 /// Process SILModule's statistics changes.
@@ -867,10 +933,10 @@ void OptimizerStatsAnalysis::updateModuleStats(TransformationContext &Ctx) {
       auto &FuncStat = getFunctionStat(&F);
       FunctionStat NewFuncStat(&F);
       processFuncStatHistory(&F, NewFuncStat, Ctx);
-      // Update function stats.
-      FuncStat = NewFuncStat;
       // Update module stats.
       NewModStat.addFunctionStat(NewFuncStat);
+      // Update function stats.
+      FuncStat = std::move(NewFuncStat);
     }
   } else {
     // Go only over functions that were changed since the last computation.
@@ -893,20 +959,20 @@ void OptimizerStatsAnalysis::updateModuleStats(TransformationContext &Ctx) {
       auto *F = InvalidatedFuncs.back();
       InvalidatedFuncs.pop_back();
       auto &FuncStat = getFunctionStat(F);
-      auto OldFuncStat = FuncStat;
+      auto &OldFuncStat = FuncStat;
       FunctionStat NewFuncStat(F);
       processFuncStatsChanges(F, OldFuncStat, NewFuncStat, Ctx);
       NewModStat.subFunctionStat(OldFuncStat);
       NewModStat.addFunctionStat(NewFuncStat);
-      FuncStat = NewFuncStat;
+      FuncStat = std::move(NewFuncStat);
     }
 
     // Process deleted functions.
     while (!DeletedFuncs.empty()) {
       auto *F = DeletedFuncs.back();
       DeletedFuncs.pop_back();
       auto &FuncStat = getFunctionStat(F);
-      auto OldFuncStat = FuncStat;
+      auto &OldFuncStat = FuncStat;
       FunctionStat NewFuncStat;
       processFuncStatsChanges(F, OldFuncStat, NewFuncStat, Ctx);
       NewModStat.subFunctionStat(OldFuncStat);
@@ -922,7 +988,7 @@ void OptimizerStatsAnalysis::updateModuleStats(TransformationContext &Ctx) {
       FunctionStat NewFuncStat(F);
       processFuncStatsChanges(F, OldFuncStat, NewFuncStat, Ctx);
       NewModStat.addFunctionStat(NewFuncStat);
-      FuncStat = NewFuncStat;
+      FuncStat = std::move(NewFuncStat);
     }
   }
 
@@ -939,7 +1005,7 @@ void OptimizerStatsAnalysis::updateModuleStats(TransformationContext &Ctx) {
 }
 
 FunctionStat::FunctionStat(SILFunction *F) {
-  InstCountVisitor V(InstCounts);
+  InstCountVisitor V(InstCounts, VarNames, DebugVariables);
   V.visitSILFunction(F);
   BlockCount = V.getBlockCount();
   InstCount = V.getInstCount();
@@ -957,7 +1023,8 @@ void swift::updateSILModuleStatsAfterTransform(SILModule &M,
                                                SILTransform *Transform,
                                                SILPassManager &PM,
                                                int PassNumber, int Duration) {
-  if (!SILStatsModules && !SILStatsFunctions && !SILStatsDumpAll)
+  if (!SILStatsModules && !SILStatsFunctions && !SILStatsLostVariables
+      && !SILStatsDumpAll)
     return;
   TransformationContext Ctx(M, PM, Transform, PassNumber, Duration);
   OptimizerStatsAnalysis *Stats = PM.getAnalysis<OptimizerStatsAnalysis>();

utils/optimizer_counters_to_sql.py

@@ -92,13 +92,13 @@ def addStatsFromInput(inputFile, db):
     for line in inputFile:
         # Read next line
         # Split into segments
-        segments = line.split(",")
+        segments = line.split(", ")
         if len(segments) < 6 or not (segments[0] in [
                 'module', 'function', 'function_history']):
             continue
         # Trim all values
-        segments = map(str.strip, segments)
-        if segments[0] == 'function_history':
+        segments = list(map(str.strip, segments))
+        if segments[0] == 'function_history' or segments[1] == 'lostvars':
             # Process history records
             delta = 0.0
             (kind, counter, stage, transform, passnum,
@@ -128,7 +128,7 @@ def processStatsFile(filename, dbname):
             filename,
             dbname))
     db = OptStatsDB(dbname)
-    with open(filename, "rb") as inputFile:
+    with open(filename, "r") as inputFile:
         addStatsFromInput(inputFile, db)
     db.finish()
 

utils/process-stats-lost-variables

@@ -0,0 +1,24 @@
+#!/usr/bin/awk -f
+
+# This source file is part of the Swift.org open source project
+#
+# Copyright (c) 2014 - 2020 Apple Inc. and the Swift project authors
+# Licensed under Apache License v2.0 with Runtime Library Exception
+#
+# See https://swift.org/LICENSE.txt for license information
+# See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
+
+BEGIN {
+  FS = ", "
+  OFS = ", "
+  print "Pass Name", "Lost Variables"
+}
+
+$1 == "function" && $2 == "lostvars" {
+  stats[$4] += $6
+}
+
+END {
+  for (i in stats)
+    print i, stats[i]
+}