-
Notifications
You must be signed in to change notification settings - Fork 15.2k
[llvm] add support for mustache templating language #105893
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 11 commits
Commits
Show all changes
87 commits
Select commit
Hold shift + click to select a range
ac1c7b5
[clang-doc] add suport for clang-doc enum generation
PeterChou1 bcc4b0d
[clang-doc] remove useless code
PeterChou1 5fe47ca
[clang-doc] modify unittest
PeterChou1 28fb40f
[clang-doc] address pr comments
PeterChou1 0d150ea
[clang-doc] revert CommentInfo change
PeterChou1 e5e70b8
[clang-doc] fix test
PeterChou1 1c4f631
[clang-doc] fix unittest
PeterChou1 b8f3f9c
[clang-doc] address pr comments
PeterChou1 bb98aad
[clang-doc] clang-format
PeterChou1 c60e2b5
[llvm] implement support for mustache template language
PeterChou1 0a20de8
Merge branch 'main' into llvm-add-mustache
PeterChou1 8290c38
[llvm][Support] Finish implementation of Mustache
PeterChou1 07d31b4
Merge branch 'llvm-add-mustache' of https://github.com/PeterChou1/llv…
PeterChou1 6e893c0
[llvm][support] fix mustache test
PeterChou1 976593e
[llvm][support] add comments
PeterChou1 6a60eab
[llvm][support] add comments
PeterChou1 6a1fcc8
[llvm][support] use enumerate for loop
PeterChou1 f1c27af
Merge branch 'main' into llvm-add-mustache
PeterChou1 eb1e1a6
[llvm][support] clang-format
PeterChou1 8ff1100
Merge branch 'llvm-add-mustache' of https://github.com/PeterChou1/llv…
PeterChou1 f4b0520
[llvm][support] fix mustache test
PeterChou1 7ffaeec
[llvm][support] clang-format
PeterChou1 746fb97
[llvm][support] use llvm enumerate
PeterChou1 4944435
Merge branch 'main' into llvm-add-mustache
PeterChou1 bcc86fe
[llvm][support] fix unittest
PeterChou1 2ceb0b0
Merge branch 'llvm-add-mustache' of https://github.com/PeterChou1/llv…
PeterChou1 95ad7a6
[llvm][support] clang-format
PeterChou1 bb3b1ac
[llvm][support] address mustache comments
PeterChou1 d8aa85c
[llvm][support] clang-format
PeterChou1 0534a05
[llvm][support] clang-format
PeterChou1 06da7a5
[llvm][support] clang-format
PeterChou1 f713198
[llvm] mustache address comments
PeterChou1 6b4f5cd
[llvm] clang-format
PeterChou1 d400c29
[llvm] fix errors
PeterChou1 8fa5fd7
[llvm] fix more bugs
PeterChou1 fd7c106
[llvm] change mustache to pass by reference
PeterChou1 29bba68
[llvm] fix failing mustache regression test
PeterChou1 7eed82f
[llvm] format
PeterChou1 e73454d
[llvm] remove unused mustache member
PeterChou1 b58fbcb
[llvm] remove unused mustache member
PeterChou1 bb4ba40
[llvm] address more comments
PeterChou1 b0ce196
[llvm] address comments
PeterChou1 561c7eb
clang-format
PeterChou1 96f6990
clang-format
PeterChou1 a247423
[llvm] factor out internal classes
PeterChou1 a0e7d48
[llvm] clang-format
PeterChou1 4165fec
[llvm] refactor mustache to raw_ostream
PeterChou1 5b0a20a
[llvm] refactor to use os_stream
PeterChou1 ba4a6db
[llvm] clang-format
PeterChou1 362728f
[llvm] fix indentation bug
PeterChou1 74d69b3
address feedback
PeterChou1 5281dc4
clang format
PeterChou1 569df38
update
PeterChou1 91547a5
address more feedback
PeterChou1 a50461b
add static keyword
PeterChou1 e596a5a
rename mustache string
PeterChou1 7eb288d
format
PeterChou1 b0f2f02
address more feedback
PeterChou1 adb2534
address more feedback
PeterChou1 0bc60a6
format
PeterChou1 a97beca
formatting
PeterChou1 40cbec9
fix grammar
PeterChou1 e8bd889
format
PeterChou1 91d4217
formatting
PeterChou1 1198d3d
address more feedback
PeterChou1 ba8ee26
format code
PeterChou1 9895f9d
address feedback
PeterChou1 03ad2f4
format
PeterChou1 3649ae6
minor change
PeterChou1 72343f4
rename variable
PeterChou1 4729743
rename variable
PeterChou1 a780958
address more pr comments
PeterChou1 c9f0cc6
add more tests
PeterChou1 0c9bc1c
Merge branch 'main' into llvm-add-mustache
PeterChou1 dc49fd7
use std::move
PeterChou1 c6a9935
Merge branch 'llvm-add-mustache' of https://github.com/PeterChou1/llv…
PeterChou1 ec452cf
clang-format
PeterChou1 254bf2e
fix broken test
PeterChou1 83f95f0
typo
PeterChou1 d5f380b
address more feedback
PeterChou1 b5f43b7
clang-format
PeterChou1 f6d18b7
grammar
PeterChou1 d6bf982
clang-format
PeterChou1 1e62179
clang-format
PeterChou1 2191e67
address more PR feedback
PeterChou1 ac8d17f
formatting
PeterChou1 345012b
formatting
PeterChou1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| //===--- Mustache.h ---------------------------------------------*- C++ -*-===// | ||
| // | ||
| // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. | ||
| // See https://llvm.org/LICENSE.txt for license information. | ||
| // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception | ||
| // | ||
| //===----------------------------------------------------------------------===// | ||
| // | ||
| // Implementation of the Mustache templating language supports version 1.4.2 | ||
| // (https://mustache.github.io/mustache.5.html). | ||
| // | ||
| //===----------------------------------------------------------------------===// | ||
|
|
||
| #ifndef LLVM_SUPPORT_MUSTACHE | ||
| #define LLVM_SUPPORT_MUSTACHE | ||
|
|
||
| #include "Error.h" | ||
| #include "llvm/ADT/StringMap.h" | ||
| #include "llvm/Support/JSON.h" | ||
| #include <string> | ||
| #include <variant> | ||
| #include <vector> | ||
|
|
||
| namespace llvm { | ||
| namespace mustache { | ||
|
|
||
| using Accessor = std::vector<std::string>; | ||
|
|
||
| class Token { | ||
| public: | ||
| enum class Type { | ||
| Text, | ||
| Variable, | ||
| Partial, | ||
| SectionOpen, | ||
| SectionClose, | ||
| InvertSectionOpen, | ||
| UnescapeVariable, | ||
| Comment, | ||
| }; | ||
|
|
||
| Token(std::string Str); | ||
|
|
||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| Token(std::string Str, char Identifier); | ||
|
|
||
| std::string getTokenBody() const { return TokenBody; }; | ||
|
|
||
| Accessor getAccessor() const { return Accessor; }; | ||
|
|
||
| Type getType() const { return TokenType; }; | ||
|
|
||
| private: | ||
| Type TokenType; | ||
| Accessor Accessor; | ||
| std::string TokenBody; | ||
| }; | ||
|
|
||
| class ASTNode { | ||
| public: | ||
| enum Type { | ||
| Root, | ||
| Text, | ||
| Partial, | ||
| Variable, | ||
| UnescapeVariable, | ||
| Section, | ||
| InvertSection, | ||
| }; | ||
|
|
||
| ASTNode() : T(Type::Root), LocalContext(nullptr){}; | ||
|
|
||
| ASTNode(std::string Body, std::shared_ptr<ASTNode> Parent) | ||
| : T(Type::Text), Body(Body), Parent(Parent), LocalContext(nullptr){}; | ||
|
|
||
| // Constructor for Section/InvertSection/Variable/UnescapeVariable | ||
| ASTNode(Type T, Accessor Accessor, std::shared_ptr<ASTNode> Parent) | ||
| : T(T), Accessor(Accessor), Parent(Parent), LocalContext(nullptr), | ||
| Children({}){}; | ||
|
|
||
| void addChild(std::shared_ptr<ASTNode> Child) { | ||
| Children.emplace_back(Child); | ||
| }; | ||
|
|
||
| std::string render(llvm::json::Value Data); | ||
|
|
||
| llvm::json::Value findContext(); | ||
|
|
||
| Type T; | ||
| std::string Body; | ||
| std::weak_ptr<ASTNode> Parent; | ||
| std::vector<std::shared_ptr<ASTNode>> Children; | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| Accessor Accessor; | ||
| llvm::json::Value LocalContext; | ||
| }; | ||
|
|
||
| class Template { | ||
| public: | ||
| static Expected<Template> createTemplate(std::string TemplateStr); | ||
|
|
||
| std::string render(llvm::json::Value Data); | ||
|
|
||
| private: | ||
| Template(std::shared_ptr<ASTNode> Tree) : Tree(Tree){}; | ||
| std::shared_ptr<ASTNode> Tree; | ||
| }; | ||
|
|
||
| } // namespace mustache | ||
| } // end namespace llvm | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| #endif // LLVM_SUPPORT_MUSTACHE | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,276 @@ | ||
| //===-- Mustache.cpp ------------------------------------------------------===// | ||
| // | ||
| // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. | ||
| // See https://llvm.org/LICENSE.txt for license information. | ||
| // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception | ||
| // | ||
| //===----------------------------------------------------------------------===// | ||
|
|
||
| #include "llvm/Support/Mustache.h" | ||
| #include "llvm/Support/Error.h" | ||
| #include <iostream> | ||
| #include <regex> | ||
| #include <sstream> | ||
|
|
||
| using namespace llvm; | ||
| using namespace llvm::json; | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| using namespace llvm::mustache; | ||
|
|
||
| std::string escapeHtml(const std::string &Input) { | ||
| DenseMap<char, std::string> HtmlEntities = {{'&', "&"}, | ||
| {'<', "<"}, | ||
| {'>', ">"}, | ||
| {'"', """}, | ||
| {'"', "'"}}; | ||
| std::string EscapedString; | ||
| EscapedString.reserve(Input.size()); | ||
|
|
||
| for (char C : Input) { | ||
| if (HtmlEntities.find(C) != HtmlEntities.end()) { | ||
| EscapedString += HtmlEntities[C]; | ||
| } else { | ||
| EscapedString += C; | ||
| } | ||
| } | ||
|
|
||
| return EscapedString; | ||
| } | ||
|
|
||
| std::vector<std::string> split(const std::string &Str, char Delimiter) { | ||
| std::vector<std::string> Tokens; | ||
| std::string Token; | ||
| std::stringstream SS(Str); | ||
| if (Str == ".") { | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| Tokens.push_back(Str); | ||
| return Tokens; | ||
| } | ||
| while (std::getline(SS, Token, Delimiter)) { | ||
| Tokens.push_back(Token); | ||
| } | ||
| return Tokens; | ||
| } | ||
|
|
||
| Token::Token(std::string Str, char Identifier) { | ||
| switch (Identifier) { | ||
| case '#': | ||
| TokenType = Type::SectionOpen; | ||
| break; | ||
| case '/': | ||
| TokenType = Type::SectionClose; | ||
| break; | ||
| case '^': | ||
| TokenType = Type::InvertSectionOpen; | ||
| break; | ||
| case '!': | ||
| TokenType = Type::Comment; | ||
| break; | ||
| case '>': | ||
| TokenType = Type::Partial; | ||
| break; | ||
| case '&': | ||
| TokenType = Type::UnescapeVariable; | ||
| break; | ||
| default: | ||
| TokenType = Type::Variable; | ||
| } | ||
| if (TokenType == Type::Comment) | ||
| return; | ||
|
|
||
| TokenBody = Str; | ||
| std::string AccessorStr = Str; | ||
| if (TokenType != Type::Variable) { | ||
| AccessorStr = Str.substr(1); | ||
| } | ||
| Accessor = split(StringRef(AccessorStr).trim().str(), '.'); | ||
| } | ||
|
|
||
| Token::Token(std::string Str) | ||
| : TokenType(Type::Text), TokenBody(Str), Accessor({}) {} | ||
|
|
||
| std::vector<Token> tokenize(std::string Template) { | ||
| std::vector<Token> Tokens; | ||
| std::regex Re(R"(\{\{(.*?)\}\})"); | ||
| std::sregex_token_iterator Iter(Template.begin(), Template.end(), Re, | ||
| {-1, 0}); | ||
| std::sregex_token_iterator End; | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| for (; Iter != End; ++Iter) { | ||
| if (!Iter->str().empty()) { | ||
| std::string Token = *Iter; | ||
| std::smatch Match; | ||
| if (std::regex_match(Token, Match, Re)) { | ||
| std::string Group = Match[1]; | ||
| Tokens.emplace_back(Group, Group[0]); | ||
| } else { | ||
| Tokens.emplace_back(Token); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return Tokens; | ||
| } | ||
|
|
||
| class Parser { | ||
| public: | ||
| Parser(std::string TemplateStr) : TemplateStr(TemplateStr) {} | ||
|
|
||
| std::shared_ptr<ASTNode> parse(); | ||
|
|
||
| private: | ||
| void parseMustache(std::shared_ptr<ASTNode> Parent); | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| std::vector<Token> Tokens; | ||
| std::size_t CurrentPtr; | ||
| std::string TemplateStr; | ||
| }; | ||
|
|
||
| std::shared_ptr<ASTNode> Parser::parse() { | ||
| Tokens = tokenize(TemplateStr); | ||
| CurrentPtr = 0; | ||
| std::shared_ptr<ASTNode> Root = std::make_shared<ASTNode>(); | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| parseMustache(Root); | ||
| return Root; | ||
| } | ||
|
|
||
| void Parser::parseMustache(std::shared_ptr<ASTNode> Parent) { | ||
|
|
||
PeterChou1 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| while (CurrentPtr < Tokens.size()) { | ||
| Token CurrentToken = Tokens[CurrentPtr]; | ||
| CurrentPtr++; | ||
| Accessor A = CurrentToken.getAccessor(); | ||
| std::shared_ptr<ASTNode> CurrentNode; | ||
|
|
||
| switch (CurrentToken.getType()) { | ||
| case Token::Type::Text: { | ||
| CurrentNode = | ||
| std::make_shared<ASTNode>(CurrentToken.getTokenBody(), Parent); | ||
| Parent->addChild(CurrentNode); | ||
PeterChou1 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| break; | ||
| } | ||
| case Token::Type::Variable: { | ||
| CurrentNode = std::make_shared<ASTNode>(ASTNode::Variable, A, Parent); | ||
| Parent->addChild(CurrentNode); | ||
| break; | ||
| } | ||
| case Token::Type::UnescapeVariable: { | ||
| CurrentNode = | ||
| std::make_shared<ASTNode>(ASTNode::UnescapeVariable, A, Parent); | ||
| Parent->addChild(CurrentNode); | ||
| break; | ||
| } | ||
| case Token::Type::Partial: { | ||
| CurrentNode = std::make_shared<ASTNode>(ASTNode::Partial, A, Parent); | ||
| Parent->addChild(CurrentNode); | ||
| break; | ||
| } | ||
| case Token::Type::SectionOpen: { | ||
| CurrentNode = std::make_shared<ASTNode>(ASTNode::Section, A, Parent); | ||
| parseMustache(CurrentNode); | ||
| Parent->addChild(CurrentNode); | ||
| break; | ||
| } | ||
| case Token::Type::InvertSectionOpen: { | ||
| CurrentNode = | ||
| std::make_shared<ASTNode>(ASTNode::InvertSection, A, Parent); | ||
| parseMustache(CurrentNode); | ||
| Parent->addChild(CurrentNode); | ||
| break; | ||
| } | ||
| case Token::Type::SectionClose: { | ||
| return; | ||
| } | ||
| default: | ||
| break; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| Expected<Template> Template::createTemplate(std::string TemplateStr) { | ||
| Parser P = Parser(TemplateStr); | ||
| Expected<std::shared_ptr<ASTNode>> MustacheTree = P.parse(); | ||
| if (!MustacheTree) | ||
| return MustacheTree.takeError(); | ||
| return Template(MustacheTree.get()); | ||
| } | ||
| std::string Template::render(Value Data) { return Tree->render(Data); } | ||
|
|
||
| std::string printJson(Value &Data) { | ||
| if (Data.getAsNull().has_value()) { | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| return ""; | ||
| } | ||
| if (auto *Arr = Data.getAsArray()) { | ||
| if (Arr->empty()) { | ||
| return ""; | ||
| } | ||
| } | ||
| if (Data.getAsString().has_value()) { | ||
| return Data.getAsString()->str(); | ||
| } | ||
| return llvm::formatv("{0:2}", Data); | ||
| } | ||
|
|
||
| std::string ASTNode::render(Value Data) { | ||
| LocalContext = Data; | ||
| Value Context = T == Root ? Data : findContext(); | ||
| switch (T) { | ||
| case Root: { | ||
| std::string Result = ""; | ||
| for (std::shared_ptr<ASTNode> Child : Children) { | ||
| Result += Child->render(Context); | ||
| } | ||
| return Result; | ||
| } | ||
| case Text: | ||
| return escapeHtml(Body); | ||
| case Partial: | ||
| break; | ||
| case Variable: | ||
| return escapeHtml(printJson(Context)); | ||
| case UnescapeVariable: | ||
| return printJson(Context); | ||
| case Section: | ||
| break; | ||
| case InvertSection: | ||
| break; | ||
| } | ||
|
|
||
| return std::string(); | ||
| } | ||
|
|
||
| Value ASTNode::findContext() { | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| if (Accessor.empty()) { | ||
| return nullptr; | ||
| } | ||
| if (Accessor[0] == ".") { | ||
| return LocalContext; | ||
| } | ||
| json::Object *CurrentContext = LocalContext.getAsObject(); | ||
| std::string &CurrentAccessor = Accessor[0]; | ||
| std::weak_ptr<ASTNode> CurrentParent = Parent; | ||
|
|
||
| while (!CurrentContext || !CurrentContext->get(CurrentAccessor)) { | ||
| if (auto Ptr = CurrentParent.lock()) { | ||
| CurrentContext = Ptr->LocalContext.getAsObject(); | ||
| CurrentParent = Ptr->Parent; | ||
| continue; | ||
| } | ||
| return nullptr; | ||
| } | ||
| Value Context = nullptr; | ||
| for (std::size_t i = 0; i < Accessor.size(); i++) { | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| CurrentAccessor = Accessor[i]; | ||
| Value *CurrentValue = CurrentContext->get(CurrentAccessor); | ||
| if (!CurrentValue) { | ||
| return nullptr; | ||
| } | ||
PeterChou1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| if (i < Accessor.size() - 1) { | ||
| CurrentContext = CurrentValue->getAsObject(); | ||
| if (!CurrentContext) { | ||
| return nullptr; | ||
| } | ||
| } else { | ||
| Context = *CurrentValue; | ||
| } | ||
| } | ||
| return Context; | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.