Skip to content

Commit dd6f188

Browse files
prestonvasquezaddaleaxbenjirewismcasimir
authored
feat(export-to-language): Add Go Support (#2991)
* Add bson-methods * Add bson * Add syntax * Add partial * Remove closures for timestamp comp * Add DeclarationStore to transpiler * Add documentaiton for DeclarationStore * remove unecessary comments * clean up generateCall comments * correct the objectID example * correct the objectID example * Revert "correct the objectID example" This reverts commit 1f58923. * Add non-idiomatic * Update packages/bson-transpilers/codegeneration/DeclarationStore.js Co-authored-by: Anna Henningsen <[email protected]> * Update packages/bson-transpilers/README.md Co-authored-by: Anna Henningsen <[email protected]> * add ruby to output languages in README.md * Finish non-idiomatic * Add imports * add state to the transpiler class * Clean up README.md * Clean up README.md * Clean up README.md * Prevent re-declaration * require template string when adding declarations * update readme * make comments uniform * make store a hash * change mock to candidate * remove duplicate method * fix more gramatical errors * add comment to candidate explanation * use bind instead of passing new args to existing templates * update readme * clean up how state is passed to bind * Add driver template * Add addFunc option to declaration store * fix typo * clean up syntax * Clean everything up * prepend variables with declarations * fix imports * add extra line to driver syntax * add extra line to non-idiomatic * uncomment the shell and python input for non-idiomatic * remove unecessary comment * add the ObjectId-Util test back * add rust and go to transpiler readme * Update packages/bson-transpilers/symbols/go/templates.yaml Co-authored-by: Benjamin Rewis <[email protected]> * Update packages/bson-transpilers/test/yaml/bson.yaml Co-authored-by: Benjamin Rewis <[email protected]> * Update packages/bson-transpilers/test/yaml/driver-syntax.yaml Co-authored-by: Benjamin Rewis <[email protected]> * clean syntax * update the driver docs comment for go * Add new line to go/templates.yaml * add native tests * replace () with {} for primitive.Regex * remove compass-schema/lib * remove compass metrics * remove compass-explain-plan/lib * remove compass-indexes/lib * remove compass-collection-stats/lib Co-authored-by: Anna Henningsen <[email protected]> Co-authored-by: Benjamin Rewis <[email protected]> Co-authored-by: Maurizio Casimirri <[email protected]>
1 parent 44c7139 commit dd6f188

File tree

29 files changed

+2483
-35
lines changed

29 files changed

+2483
-35
lines changed

‎packages/bson-transpilers/README.md‎

Lines changed: 52 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,7 @@
33
[![downloads][5]][6]
44

55
Transpilers for building BSON documents in any language. Current support
6-
provided for `shell` `javascript` and `python` as inputs. `java`, `c#`, `node`, `shell` and `python` as
7-
outputs.
6+
provided for `shell` `javascript` and `python` as inputs. `java`, `c#`, `node`, `shell`, `python`, `ruby` and `go` as outputs.
87

98
> ⚠️&nbsp;&nbsp;`shell` output produces code that is compatible only with legacy `mongo` shell not the new `mongosh` shell. See [COMPASS-4930](https://jira.mongodb.org/browse/COMPASS-4930) for some additional context
109
@@ -59,6 +58,57 @@ Any transpiler errors that occur will be thrown. To catch them, wrap the
5958
- __error.column:__ If it is a syntax error, will have the column.
6059
- __error.symbol:__ If it is a syntax error, will have the symbol associated with the error.
6160

61+
62+
### State
63+
64+
The `CodeGenerationVisitor` class manages a global state which is bound to the `argsTemplate` functions. This state is intended to be used as a solution for the `argsTemplate` functions to communicate with the `DriverTemplate` function. For example:
65+
66+
```yaml
67+
ObjectIdEqualsArgsTemplate: &ObjectIdEqualsArgsTemplate !!js/function >
68+
(_) => {
69+
this.oneLineStatement = "Hello World";
70+
return '';
71+
}
72+
73+
DriverTemplate: &DriverTemplate !!js/function >
74+
(_spec) => {
75+
return this.oneLineStatement;
76+
}
77+
```
78+
79+
The output of the driver syntax for this language will be the one-line statement `Hello World`.
80+
81+
#### DeclarationStore
82+
A more practical use-case of state is to accumulate variable declarations throughout the `argsTemplate` to be rendered by the `DriverTemplate`. That is, the motivation for using `DeclarationStore` is to prepend the driver syntax with variable declarations rather than using non-idiomatic solutions such as closures.
83+
84+
The `DeclarationStore` class maintains an internal state concerning variable declarations. For example,
85+
86+
```javascript
87+
// within the args template
88+
(arg) => {
89+
return this.declarations.add("Temp", "objectID", (varName) => {
90+
return [
91+
`${varName}, err := primitive.ObjectIDFromHex(${arg})`,
92+
'if err != nil {',
93+
' log.Fatal(err)',
94+
'}'
95+
].join('\n')
96+
})
97+
}
98+
```
99+
100+
Note that each use of the same variable name will result in an increment being added to the declaration statement. For example, if the variable name `objectIDForTemp` is used two times the resulting declaration statements will use `objectIDForTemp` for the first declaration and `objectID2ForTemp` for the second declaration. The `add` method returns the incremented variable name, and is therefore what would be expected as the right-hand side of the statement defined by the `argsTemplate` function.
101+
102+
The instance of the `DeclarationStore` constructed by the transpiler class is passed into the driver, syntax via state, for use:
103+
104+
```javascript
105+
(spec) => {
106+
const comment = '// some comment'
107+
const client = 'client, err := mongo.Connect(context.Background(), options.Client().ApplyURI(cs.String()))'
108+
return "#{comment}\n\n#{client}\n\n${this.declarations.toString()}"
109+
}
110+
```
111+
62112
### Errors
63113
There are a few different error classes thrown by `bson-transpilers`, each with
64114
their own error code:

‎packages/bson-transpilers/codegeneration/CodeGenerationVisitor.js‎

Lines changed: 26 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ const {
1010
} = require('../helper/error');
1111

1212
const { removeQuotes } = require('../helper/format');
13+
const DeclarationStore = require('./DeclarationStore');
1314

1415
/**
1516
* Class for code generation. Goes in between ANTLR generated visitor and
@@ -24,6 +25,7 @@ module.exports = (ANTLRVisitor) => class CodeGenerationVisitor extends ANTLRVisi
2425
super();
2526
this.idiomatic = true; // PUBLIC
2627
this.clearImports();
28+
this.state = { declarations: new DeclarationStore() };
2729
}
2830

2931
clearImports() {
@@ -53,15 +55,32 @@ module.exports = (ANTLRVisitor) => class CodeGenerationVisitor extends ANTLRVisi
5355
return this[rule](ctx);
5456
}
5557

58+
returnResultWithDeclarations(ctx) {
59+
let result = this.returnResult(ctx);
60+
if (this.getState().declarations.length() > 0) {
61+
result = `${this.getState().declarations.toString() + '\n\n'}${result}`;
62+
}
63+
return result;
64+
}
65+
5666
/**
5767
* PUBLIC: This is the entry point for the compiler. Each visitor must define
5868
* an attribute called "startNode".
5969
*
6070
* @param {ParserRuleContext} ctx
71+
* @param {Boolean} useDeclarations - prepend the result string with declarations
6172
* @return {String}
6273
*/
63-
start(ctx) {
64-
return this.returnResult(ctx).trim();
74+
start(ctx, useDeclarations = false) {
75+
return (useDeclarations ? this.returnResultWithDeclarations(ctx) : this.returnResult(ctx)).trim();
76+
}
77+
78+
getState() {
79+
return this.state;
80+
}
81+
82+
clearDeclarations() {
83+
this.getState().declarations.clear();
6584
}
6685

6786
/**
@@ -330,7 +349,7 @@ module.exports = (ANTLRVisitor) => class CodeGenerationVisitor extends ANTLRVisi
330349
}
331350
returnFunctionCallLhsRhs(lhs, rhs, lhsType, l) {
332351
if (lhsType.argsTemplate) {
333-
rhs = lhsType.argsTemplate(l, ...rhs);
352+
rhs = lhsType.argsTemplate.bind(this.getState())(l, ...rhs);
334353
} else {
335354
rhs = `(${rhs.join(', ')})`;
336355
}
@@ -494,7 +513,7 @@ module.exports = (ANTLRVisitor) => class CodeGenerationVisitor extends ANTLRVisi
494513
? lhsType.template()
495514
: defaultT;
496515
const rhs = lhsType.argsTemplate
497-
? lhsType.argsTemplate(lhsArg, ...args)
516+
? lhsType.argsTemplate.bind(this.getState())(lhsArg, ...args)
498517
: defaultA;
499518
const lhs = skipLhs ? '' : lhsArg;
500519
return this.Syntax.new.template
@@ -516,7 +535,7 @@ module.exports = (ANTLRVisitor) => class CodeGenerationVisitor extends ANTLRVisi
516535
let args = '';
517536
const keysAndValues = this.getKeyValueList(ctx);
518537
if (ctx.type.argsTemplate) {
519-
args = ctx.type.argsTemplate(
538+
args = ctx.type.argsTemplate.bind(this.getState())(
520539
this.getKeyValueList(ctx).map((k) => {
521540
return [this.getKeyStr(k), this.visit(this.getValue(k))];
522541
}),
@@ -526,7 +545,7 @@ module.exports = (ANTLRVisitor) => class CodeGenerationVisitor extends ANTLRVisi
526545
}
527546
ctx.indentDepth--;
528547
if (ctx.type.template) {
529-
return ctx.type.template(args, ctx.indentDepth);
548+
return ctx.type.template.bind(this.getState())(args, ctx.indentDepth);
530549
}
531550
return this.visitChildren(ctx);
532551
}
@@ -545,7 +564,7 @@ module.exports = (ANTLRVisitor) => class CodeGenerationVisitor extends ANTLRVisi
545564
if (ctx.type.argsTemplate) { // NOTE: not currently being used anywhere.
546565
args = visitedElements.map((arg, index) => {
547566
const last = !visitedElements[index + 1];
548-
return ctx.type.argsTemplate(arg, ctx.indentDepth, last);
567+
return ctx.type.argsTemplate.bind(this.getState())(arg, ctx.indentDepth, last);
549568
}).join('');
550569
} else {
551570
args = visitedElements.join(', ');

‎packages/bson-transpilers/codegeneration/DeclarationStore.js‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
/**
2+
* Stores declarations for use in the DriverTemplate
3+
*
4+
* @returns {object}
5+
*/
6+
class DeclarationStore {
7+
constructor() {
8+
this.clear();
9+
}
10+
11+
/**
12+
* Add declarations by templateID + varRoot + declaration combo. Duplications will not be collected, rather the add
13+
* method will return the existing declaration's variable name.
14+
*
15+
* @param {string} templateID - Name/alias of the template the declaration is being made for
16+
* @param {string} varRoot - The root of the variable name to be appended by the occurance count
17+
* @param {function} declaration - The code block to be prepended to the driver syntax
18+
* @returns {string} the variable name with root and appended count
19+
*/
20+
addVar(templateID, varRoot, declaration) {
21+
// Don't push existing declarations
22+
const current = this.alreadyDeclared(templateID, varRoot, declaration);
23+
if (current !== undefined) {
24+
return current;
25+
}
26+
const varName = this.next(templateID, varRoot);
27+
this.vars[declaration(varName)] = varName;
28+
return varName;
29+
}
30+
31+
/**
32+
* Add a function to the funcs set
33+
*
34+
* @param {string} fn - String literal of a function
35+
*/
36+
addFunc(fn) {
37+
if (!this.funcs[fn]) {
38+
this.funcs[fn] = true;
39+
}
40+
}
41+
42+
alreadyDeclared(templateID, varRoot, declaration) {
43+
const existing = this.candidates(templateID, varRoot);
44+
for (var i = 0; i < existing.length; i++) {
45+
const candidate = `${this.varTemplateRoot(templateID, varRoot)}${i > 0 ? i : ''}`;
46+
const current = this.vars[declaration(candidate)];
47+
if (current !== undefined) {
48+
return current;
49+
}
50+
}
51+
}
52+
53+
candidates(templateID, varRoot) {
54+
const varTemplateRoot = this.varTemplateRoot(templateID, varRoot);
55+
return Object.values(this.vars).filter(varName => varName.startsWith(varTemplateRoot));
56+
}
57+
58+
clear() {
59+
this.vars = {};
60+
this.funcs = {};
61+
}
62+
63+
length() {
64+
return Object.keys(this.vars).length + Object.keys(this.funcs).length;
65+
}
66+
67+
next(templateID, varRoot) {
68+
const existing = this.candidates(templateID, varRoot);
69+
70+
// If the data does not exist in the vars, then the count should append nothing to the variable name
71+
const count = existing.length > 0 ? existing.length : '';
72+
return `${this.varTemplateRoot(templateID, varRoot)}${count}`;
73+
}
74+
75+
/**
76+
* Stringify the variable declarations
77+
*
78+
* @param {string} sep - Seperator string placed between elements in the resulting string of declarations
79+
* @returns {string} all the declarations as a string seperated by a line-break
80+
*/
81+
toString(sep = '\n\n') {
82+
return [...Object.keys(this.vars), ...Object.keys(this.funcs)].join(sep);
83+
}
84+
85+
varTemplateRoot(templateID, varRoot) {
86+
return `${varRoot}${templateID}`;
87+
}
88+
}
89+
90+
module.exports = DeclarationStore;

‎packages/bson-transpilers/codegeneration/go/Generator.js‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
/*
2+
* Class for handling edge cases for Go code generation. Defines "emit" methods.
3+
*/
4+
module.exports = (Visitor) => class Generator extends Visitor {
5+
constructor() { super(); }
6+
};

‎packages/bson-transpilers/codegeneration/object/Generator.js‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ module.exports = (Visitor) => class Generator extends Visitor {
9292
}
9393

9494
if (lhsType && lhsType.argsTemplate) {
95-
return lhsType.argsTemplate(lhs, ...args);
95+
return lhsType.argsTemplate.bind(this.getState())(lhs, ...args);
9696
}
9797

9898
let expr;
@@ -163,7 +163,7 @@ module.exports = (Visitor) => class Generator extends Visitor {
163163
return this.Syntax.equality.template(s, op, this.visit(arr[i + 1]));
164164
}
165165
if (op === 'in' || op === 'notin') {
166-
return this.Syntax.in.template(s, op, this.visit(arr[i + 1]));
166+
return this.Syntax.in.template.bind(this.state)(s, op, this.visit(arr[i + 1]));
167167
}
168168
throw new BsonTranspilersRuntimeError(`Unrecognized operation ${op}`);
169169
}, this.visit(ctx.children[0]));

‎packages/bson-transpilers/codegeneration/python/Visitor.js‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -171,7 +171,7 @@ module.exports = (CodeGenerationVisitor) => class Visitor extends CodeGeneration
171171
if (ctx.type.argsTemplate) { // NOTE: not currently being used anywhere.
172172
args = visitedElements.map((arg, index) => {
173173
const last = !visitedElements[index + 1];
174-
return ctx.type.argsTemplate(arg, ctx.indentDepth, last);
174+
return ctx.type.argsTemplate.bind(this.getState())(arg, ctx.indentDepth, last);
175175
});
176176
join = '';
177177
} else {
@@ -338,7 +338,7 @@ module.exports = (CodeGenerationVisitor) => class Visitor extends CodeGeneration
338338
if (op === 'in' || op === 'notin') {
339339
skip = true;
340340
if (this.Syntax.in) {
341-
return `${str}${this.Syntax.in.template(
341+
return `${str}${this.Syntax.in.template.bind(this.state)(
342342
this.visit(arr[i - 1]), op, this.visit(arr[i + 1]))}`;
343343
}
344344
return `${str} ${this.visit(arr[i - 1])} ${op} ${this.visit(arr[i + 1])}`;

‎packages/bson-transpilers/compile-symbol-table.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ const loadSymbolTable = (dir, inputLang, outputLang) => {
5959
const loadAll = () => {
6060
const dir = path.join(__dirname, 'lib', 'symbol-table');
6161
const inputLangs = ['javascript', 'shell', 'python'];
62-
const outputLangs = ['java', 'shell', 'python', 'csharp', 'javascript', 'object', 'ruby', 'rust'];
62+
const outputLangs = ['java', 'shell', 'python', 'csharp', 'javascript', 'object', 'ruby', 'go', 'rust'];
6363
if (!fs.existsSync(dir)) {
6464
fs.mkdirSync(dir);
6565
}

0 commit comments

Comments
 (0)