Skip to content

[VPlan] Sink retrieving legacy costs to more specific computeCost impls. #109708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Oct 9, 2024
+65 −31
Merged

[VPlan] Sink retrieving legacy costs to more specific computeCost impls. #109708

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8117ba1
[VPlan] Return Max from computeCost without underlying instrs.
fhahn Sep 23, 2024
f409232
Merge remote-tracking branch 'origin/main' into vplan-max-cost-sentinel
fhahn Oct 6, 2024
0bf2760
!fixup address latest comments, thanks!
fhahn Oct 6, 2024
9f12dab
Merge remote-tracking branch 'origin/main' into vplan-max-cost-sentinel
fhahn Oct 8, 2024
16ca2ca
Merge remote-tracking branch 'origin/main' into vplan-max-cost-sentinel
fhahn Oct 8, 2024
f632630
!fixup rework
fhahn Oct 8, 2024
c75d678
Merge remote-tracking branch 'origin/main' into vplan-max-cost-sentinel
fhahn Oct 9, 2024
d32eb48
!Fixup address latest comments, thanks!
fhahn Oct 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion llvm/lib/Transforms/Vectorize/VPlan.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -844,7 +844,9 @@ class VPRecipeBase : public ilist_node_with_parent<VPRecipeBase, VPBasicBlock>,
protected:
/// Compute the cost of this recipe either using a recipe's specialized
/// implementation or using the legacy cost model and the underlying
/// instructions.
/// instructions. Returns InstructionCost::max() if the cost of this recipe
Copy link
Collaborator

@ayalz ayalz Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The underlying instruction can be passed as argument, except that this base implementation should be a default for all derived VPlan-based computations, where the latter should be independent on UI, if possible?

Copy link
Contributor Author

@fhahn fhahn Oct 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, also there are some computeCost implementation for recipes with underlying instruction, for which the default should not trigger. In particular, all recipes which currently have computeCost implemented in subclasses should be included with force-target-instruction-cost.

Copy link
Collaborator

@ayalz ayalz Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively computeCost could return an optional cost.

Perhaps better to return no cost at all, as in optional, when cost should be ignored, rather than a max sentinel. The latter requires additional attention to remain unmodified, preventing (mistaken) updates.

Copy link
Contributor Author

@fhahn fhahn Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change is not needed in latest version.

/// should be ignored. Forced target instruction cost is not applied for such
/// recipes.
virtual InstructionCost computeCost(ElementCount VF,
VPCostContext &Ctx) const;
};
Expand Down
20 changes: 9 additions & 11 deletions llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,14 +276,6 @@ static Instruction *getInstructionForCost(const VPRecipeBase *R) {
return dyn_cast_or_null<Instruction>(S->getUnderlyingValue());
Copy link
Collaborator

@ayalz ayalz Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're here, this could return S->getUnderlyingInstr(), if the latter does cast_if_present rather than cast?

Copy link
Contributor Author

@fhahn fhahn Oct 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I think originally the intention was to only use getUnderlyingInstr if it is guaranteed to be available, but perhaps it is time to revisit this?

Copy link
Collaborator

@ayalz ayalz Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, right. May be good to revisit this intention.

if (auto *IG = dyn_cast<VPInterleaveRecipe>(R))
return IG->getInsertPos();
// Currently the legacy cost model only calculates the instruction cost with
// underlying instruction. Removing the WidenMem here will prevent
// force-target-instruction-cost overwriting the cost of recipe with
// underlying instruction which is inconsistent with the legacy model.
// TODO: Remove WidenMem from this function when we don't need to compare to
// the legacy model.
if (auto *WidenMem = dyn_cast<VPWidenMemoryRecipe>(R))
return &WidenMem->getIngredient();
Copy link
Collaborator

@ayalz ayalz Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth leaving a note, that even though widen loads and stores have an underlying instruction ("ingredient"), null is returned - in order to match legacy cost model behavior with force-target-instruction-cost?

The instruction a recipe provides for cost affects both

  • skipCostComputation() - widen loads and stores are irrelevant for skipping?
  • force-target-instruction-cost - widen loads and stores should be assigned zero instead of this overriding cost?
  • w/o force-target-instruction-cost - should be assigned zero cost?

Copy link
Contributor Author

@fhahn fhahn Oct 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified the the comment about the return value for the function.

Nullptr is returned if either the recipe doesn't have an underlying instruction or computeCost is implemented and there is no need for the underlying instruction (i.e. it should not be needed to be skipped)

Copy link
Collaborator

@ayalz ayalz Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nullptr is returned if either the recipe doesn't have an underlying instruction or computeCost is implemented and there is no need for the underlying instruction (i.e. it should not be needed to be skipped)

Regarding the "else" part of this "or" - not needing UI because recipe is not to be skipped is fine (if it's absence from SkipCostComputation it won't be skipped), but would UI still be needed in order to apply ForceTargetInstructionCost?

Copy link
Contributor Author

@fhahn fhahn Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been reworked in the latest version. The function is gone and inline to ::cost(), which clarifies that it is only used to decide to skip and when to apply forced costs.

return nullptr;
}

Expand All @@ -293,9 +285,13 @@ InstructionCost VPRecipeBase::cost(ElementCount VF, VPCostContext &Ctx) {
return 0;
Copy link
Collaborator

@ayalz ayalz Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that skipped costs also skip debug dump.

Copy link
Contributor Author

@fhahn fhahn Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest version


InstructionCost RecipeCost = computeCost(VF, Ctx);
if (UI && ForceTargetInstructionCost.getNumOccurrences() > 0 &&
RecipeCost.isValid())
if (ForceTargetInstructionCost.getNumOccurrences() > 0 &&
(RecipeCost.isValid() && RecipeCost != InstructionCost::getMax()))
RecipeCost = InstructionCost(ForceTargetInstructionCost);
// Max cost is used as a sentinel value to detect recipes without underlying
// instructions for which no forced target instruction cost should be applied.
Copy link
Collaborator

@ayalz ayalz Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to indicate why such a sentinel is needed, rather than detecting the absence of an underlying instruction by directly checking UI, as done now?

Copy link
Contributor Author

@fhahn fhahn Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No sentinel is needed in the latest version, it uses UI now, thanks!

if (RecipeCost == InstructionCost::getMax())
Copy link
Collaborator

@ayalz ayalz Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elseif? (Otherwise a ForceTargetInstructionCost of Max will be replaced by zero.)

Copy link
Contributor Author

@fhahn fhahn Oct 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, thanks!

RecipeCost = 0;
Copy link
Collaborator

@ayalz ayalz Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So cost is essentially a valid non-overridable/un-forceable value, of zero?

Would adding, say, a "Permanent" state to CostState, along with Valid and InValid, be reasonable?

Copy link
Contributor Author

@fhahn fhahn Oct 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes exactly. Not sure if it would be worth adding a new state for this very specific use case?


LLVM_DEBUG({
dbgs() << "Cost of " << RecipeCost << " for VF " << VF << ": ";
Expand All @@ -315,7 +311,9 @@ InstructionCost VPRecipeBase::computeCost(ElementCount VF,
// transform, avoid computing their cost multiple times for now.
Ctx.SkipCostComputation.insert(UI);
}
return UI ? Ctx.getLegacyCost(UI, VF) : 0;
// Max cost is used as a sentinel value to detect recipes without underlying
// instructions for which no forced target instruction cost should be applied.
return UI ? Ctx.getLegacyCost(UI, VF) : InstructionCost::getMax();
Copy link
Collaborator

@ayalz ayalz Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment above about returning zero when no underlying instruction is there, should be updated.

Under ForceTargetInstructionCost, all UI's with valid costs are assigned ForceTargetInstructionCost cost. Therefore, under ForceTargetInstructionCost, all recipes w/ UI's and valid costs should also be assigned this cost, and all recipes w/o UI's should be assigned zero cost?

Need to distinguish between (zero, but could be any other valid) cost of a recipe that has no underlying, which should be ignored by ForceTargetInstructionCost because legacy only addresses UI's, and recipes w/ underlying (whose legacy cost is returned, can be zero or any other valid cost) which are subject to ForceTargetInstructionCost.

Copy link
Contributor Author

@fhahn fhahn Oct 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<Move the comment above to update the existing comment, thanks!

Copy link
Collaborator

@ayalz ayalz Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps the default base-class implementation of VPRecipeBase::computeCost() should return null, delegating the fallback use of Ctx.getLegacyCost(UI, VF) to its caller VPRecipeBase::cost(), as outlined above?

Copy link
Contributor Author

@fhahn fhahn Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjusted to use getLegacyCost in a few relevant sub-classes (VPSingleDefRecipe/VPInterleaveRecipe). Should hopefully be clearer and simpler now

}

FastMathFlags VPRecipeWithIRFlags::getFastMathFlags() const {
Expand Down