Chrimc/form (#384)

* Intermediate check-in before inverting
template/prop loops.

* Working generator with incomplete templates.

* Basic .lu files working.

* Update to generate single .lu file.

* Fix cli issues.

* Fix CLI issues with template resources.

* Clean up lint warnings.

* Fix lint errors.

* Generate composite .lu

* Add dialog and multi-language support.

* shortcut to debug

* Add per-locale .lg file.

* Update debug config to use temp directory.
Generate multilingual .lg file.

* Update to include schema prefix on names.

* Fix bugs in .lg genertion.

* try help

* help and cancel

* Copy schema to output directory.

* help and cancel version 1

* separate help.lu

* several fixes

* remove help enum

* reset launch.json

* remove empty line

* Add the ability to copy library dialog files.

* Add ability to specify schema and multiple
templates.

* Rename resources.
Support multiple templates.
Add *PROPN** pattern.

* Fix bug in striingify call.

* Generate schema.main.dialog and update config.

* Update templates to newest format.

* Update templates to match names.

* add .dialogs for several basic rules

* Update templates to use $ and add begindialog as a hack to work around ada
ptive issue.

* add .dialogs for several basic rules

* fix comments

* merge from origin

* small changes

* Add formCopy cmd file.

* Adjust for bug in templates.

* fix path

* Version 1 of contact.form.dialog

* Same info is modeled for both Home and Work.
* List of state abbreviations and their spelled-out names are specified in the separate state.en-us.lu, and used in the $mappings specifications.
* Latitude and Longitude should be auto-filled if the corresponding address are filled.
* Validation patterns are specified for zip codes and phone numbers.

* Fixed contact.form.dialog and added activity.form.dialog

* contact.form.dialog
  - Fixed spelling
  - Use phonenumber prebuilt entity type
* activity.form.dialog: the second scenario from Darren "An employee working for a customer wants to add a quick update to a customer record after a call. This update will then be available for other people working with that customer to see".

* Added opportunity.form.dialog

For Darren's scenario "An employee working with a customer has had a meeting and needs to create a new opportunity."

* change names

* Revert "change names"

This reverts commit 64ffaff.

* change names

* basic work done for ask, set and help

* cancel confirmation

* cancel confirmation done, unexpected property set confirmation in progress

* improve unexpected property test

* Version 1 of contact.form.dialog

* Same info is modeled for both Home and Work.
* List of state abbreviations and their spelled-out names are specified in the separate state.en-us.lu, and used in the $mappings specifications.
* Latitude and Longitude should be auto-filled if the corresponding address are filled.
* Validation patterns are specified for zip codes and phone numbers.

* Fixed contact.form.dialog and added activity.form.dialog

* contact.form.dialog
  - Fixed spelling
  - Use phonenumber prebuilt entity type
* activity.form.dialog: the second scenario from Darren "An employee working for a customer wants to add a quick update to a customer record after a call. This update will then be available for other people working with that customer to see".

* Added opportunity.form.dialog

For Darren's scenario "An employee working with a customer has had a meeting and needs to create a new opportunity."

* rename state.lu

* prebuilt lu templates

* remove choose entity

we only have chooseProperty and clarifyEntity

* Switch to event expected.

* Update templates for latest runtime and add ask for
multiple properties.

* Add **NAMES/**NAME** and generate property names.

* fix unexpected property confirmation usage

* change unexpected property action from ask to sendactivity

* all done confirmation

* Update names to support chooseProperty.

* Check-in bread value hack.

* Fix template bug around clarify.

* Update templates to use names and start numbers.

* complete the new forms

* contextual help

* activity schema working fine

* add meta info to opportunity schema

* opportunity semi-working

cannot use phonenumber;
properties not in required also got asked;
in stringset, default value is not expression, cannot get the value for @email

* three schemas working

sandwich, activity, opportunity

* contact schema working

Adding welcome message in beginDialog,
Add help message for non-enum properties,
All three CDS schemas working

* Sync changes for demo.

* Before adding mapping templates

* Getting close to E2e.

* Largely working with some template debugging.

* Update the readme.

* Update to latest expression and lg packages.
Support app schema.

* Update standard schema.

* Finish merge.

* Show metaschema.

* Fix readme typo.

* several changes (#299)

* several changes

(1) remove unused old files;
(2) change incorrect file names;
(3) add string ask templates;
(4) remove meta properties from schemas;
(5) fix an error in calculating optional properties

* fix help property bug

* Add string templates.

* a few changes (#318)

(1) fix the cancel.lu bug
(2) add datetime.lu
(2) stringSetdatetime.dialog.lg

* Add template for not understood.

* Connect to bf cli.

* Add basic role support.

* Nigao/newtemplate (#321)

* a few changes

(1) fix the cancel.lu bug
(2) add datetime.lu
(2) stringSetdatetime.dialog.lg

* autohelp for new templates

* Update to use @{} everywhere and remove auto-help
since it required post-ask action.

* Nigao/newtemplate (#323)

* a few changes

(1) fix the cancel.lu bug
(2) add datetime.lu
(2) stringSetdatetime.dialog.lg

* autohelp for new templates

* add $requires, and datetime entity sets

* add all stringSet for prebuilt entities explicitly

* change prebuilt-entity.lu.lg files

* Switch to using a triggerIntent instead of none for entities.
Cleanup filename expressions.

* Fix bug in status.

* Switch from dialog:form to dialog:generate

* Add top-level documentation.

* Update tests to use dialog:generate.

* Rename tests to generator.

* Fixing linting errors

* Generate confirmation card.

* Fix reference regex ot not match structured lg.

* Minor bug fixes.

* Update to allow multi-lingual fallback.

Author: Chris McConnell

packages/dialog/.vscode/launch.json

@@ -4,14 +4,6 @@
     // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
     "version": "0.2.0",
     "configurations": [
-        {
-            "type": "antlr-debug",
-            "request": "launch",
-            "name": "Debug Current Grammar",
-            "input": "test/commands/dialog/forms/input.txt",
-            "startRule": "doc",
-            "visualParseTree": true
-        },
         {
             "type": "node",
             "request": "launch",
@@ -64,17 +56,22 @@
         {
             "type": "node",
             "request": "launch",
-            "name": "Launch Program",
-            "program": "${workspaceFolder}\\bin\\run",
+            "name": "BotBuilder schema",
+            "program": "${workspaceFolder}/bin/run",
             "preLaunchTask": "tsc: build - tsconfig.json",
             "outFiles": [
                 "${workspaceFolder}/lib/**/*.js"
             ],
             "args": [
                 "dialog:merge",
-                "-f",
-                "c:/source/github/botbuilder-dotnet/**/*.schema"
-            ]
+                "libraries/**/*.schema",
+                "-u",
+                "-b", "master",
+                "-o", "schemas/sdk.schema",
+                "--verbose"
+            ],
+            "internalConsoleOptions": "openOnSessionStart",
+            "cwd": "c:/users/chrimc/source/repos/botbuilder-dotnet"
         },
         {
             "type": "node",
@@ -92,8 +89,6 @@
                 "${env:TEMP}/sandwich.out",
                 "--force",
                 "--verbose",
-                "--schema",
-                "https://raw.githubusercontent.com/microsoft/botbuilder-dotnet/chrimc/map/schemas/sdk.schema"
             ],
             "internalConsoleOptions": "openOnSessionStart"
         },
@@ -107,7 +102,7 @@
                 "${workspaceFolder}/lib/**/*.js"
             ],
             "args": [
-                "dialog:form",
+                "dialog:generate",
                 "${workspaceFolder}/test/commands/dialog/forms/activity.form.dialog",
                 "-o",
                 "${env:TEMP}/activity.out",
@@ -128,7 +123,7 @@
                 "${workspaceFolder}/lib/**/*.js"
             ],
             "args": [
-                "dialog:form",
+                "dialog:generate",
                 "${workspaceFolder}/test/commands/dialog/forms/contact.form.dialog",
                 "-o",
                 "${env:TEMP}/contact.out",
@@ -149,7 +144,7 @@
                 "${workspaceFolder}/lib/**/*.js"
             ],
             "args": [
-                "dialog:form",
+                "dialog:generate",
                 "${workspaceFolder}/test/commands/dialog/forms/opportunity.form.dialog",
                 "-o",
                 "${env:TEMP}/opportunity.out",

packages/dialog/src/commands/dialog/merge.ts

@@ -35,7 +35,7 @@ export default class DialogMerge extends Command {
     static flags: flags.Input<any> = {
         help: flags.help({ char: 'h' }),
         output: flags.string({ char: 'o', description: 'Output path and filename for merged schema. [default: app.schema]', default: 'app.schema', required: false }),
-        branch: flags.string({ char: 'b', description: 'The branch to use for the meta-schema component.schema.', default: '4.Future', required: false }),
+        branch: flags.string({ char: 'b', description: 'The branch to use for the meta-schema component.schema.', default: 'master', required: false }),
         update: flags.boolean({ char: 'u', description: 'Update .schema files to point the <branch> component.schema and regenerate component.schema if baseComponent.schema is present.', default: false, required: false }),
         verbose: flags.boolean({ description: 'output verbose logging of files as they are processed', default: false }),
     }

packages/dialog/src/library/dialogGenerator.ts

@@ -108,19 +108,21 @@ async function replaceAsync(str: string, re: RegExp, callback: (match: string) =
 }
 
 const RefPattern = /^\s*\[[^\]\n\r]*\]\s*$/gm
+const LocalePattern = /\.[^.]+\.lg$/
 async function processLibraryTemplates(template: string, outPath: string, templateDirs: string[], outDir: string, form: any, scope: any, force: boolean, feedback: Feedback): Promise<string> {
     if (!template.startsWith('>>> Library')) {
         return replaceAsync(template, RefPattern, async (match: string): Promise<string> => {
             let replacement = await processTemplate(match.substring(match.indexOf('[') + 1, match.indexOf(']')), templateDirs, outDir, form, scope, force, feedback, false)
+            replacement = replacement.replace(LocalePattern, '.lg')
             let local = ppath.relative(ppath.dirname(outPath), replacement)
-            return Promise.resolve(`[${ppath.basename(replacement)}](${local})`)
+            return Promise.resolve(`\n[${ppath.basename(replacement)}](${local})`)
         });
     } else {
         return Promise.resolve(template)
     }
 }
 
-type FileRef = { name: string, fullName: string, relative: string }
+type FileRef = { name: string, fallbackName: string, fullName: string, relative: string }
 function addEntry(fullPath: string, outDir: string, tracker: any): FileRef | undefined {
     let ref: FileRef | undefined
     let basename = ppath.basename(fullPath, '.dialog')
@@ -129,6 +131,7 @@ function addEntry(fullPath: string, outDir: string, tracker: any): FileRef | und
     if (!arr.find(ref => ref.name === basename)) {
         ref = {
             name: basename,
+            fallbackName: basename.replace(/\.[^.]+\.lg/, '.lg'),
             fullName: ppath.basename(fullPath),
             relative: ppath.relative(outDir, fullPath)
         }
@@ -275,7 +278,7 @@ export async function generate(
     }
 
     if (!metaSchema) {
-        metaSchema = 'https://raw.githubusercontent.com/microsoft/botbuilder-dotnet/chrimc/map/schemas/sdk.schema'
+        metaSchema = 'https://raw.githubusercontent.com/microsoft/botbuilder-dotnet/chrimc/form/schemas/sdk.schema'
     } else if (!metaSchema.startsWith('http')) {
         // Adjust relative to outDir
         metaSchema = ppath.relative(outDir, metaSchema)

packages/dialog/templates/ChooseProperty.dialog.lg

@@ -6,7 +6,8 @@
     "actions": [
         {
             "$type": "Microsoft.Ask",
-            "activity": "@{callChooseProperties()}"           
+            "activity": "@{callChooseProperties()}",
+            "expectedProperties": ["PROPERTYName"]
         }
     ]
 }

packages/dialog/templates/Main.dialog.lg

@@ -5,7 +5,7 @@
 - ```
 {
     "$schema": "@{appSchema}",
-    "$type": "Microsoft.FormDialog",
+    "$type": "Microsoft.AdaptiveDialog",
     "recognizer": "@{formName}.lu",
     "generator": "@{formName}.lg",
     "schema": "@{formName}.schema",

packages/dialog/templates/en-us/common.lg

@@ -60,11 +60,13 @@ Which value do you want for @{name(property)}?
 - ELSE:
 - 
 
+> TODO: We should enable this when where/foreach works over objects.
 # optionalProperties
-- IF: @{add(count(turn.schema.required),count(turn.schema.$expectedOnly)) < count(turn.schema.properties)}
-- Optional properties include: @{join(foreach(where(turn.form.properties, property, !contains(turn.schema.required && !contains(turn.schema.$expectedOnly, property), property)), property, name(property)), ', ')}\n
-- ELSE:
-- 
+-
+> - IF: @{add(count(turn.schema.required),count(turn.schema.$expectedOnly)) < count(turn.schema.properties)}
+> - Optional properties include: @{join(foreach(where(turn.schema.properties, property, !contains(turn.schema.required, property) && !contains(turn.schema.$expectedOnly, property)), property, name(property)), ', ')}\n
+> - ELSE:
+> - 
 
 # possibleValues(property)
 - @{foreach(turn.schema.properties[property].enum, enum, value(property, enum))}
@@ -134,8 +136,8 @@ Please choose a value for @{name(property)} from \[@{join(foreach(turn.dialogEve
 }
 ```
 
->>> TODO: It would be nice to able to include bound optional properties as well.  
->>> Once we have support in foreach for objects we can expand this.
+> TODO: It would be nice to able to include bound optional properties as well.  
+> Once we have support in foreach for objects we can expand this.
 # requiredFacts
 - @{join(foreach(dialog.requiredProperties, property, fact(property)), ', ')}
 
@@ -163,4 +165,6 @@ Please choose a value for @{name(property)} from \[@{join(foreach(turn.dialogEve
 # PropertyToChange(value)
 - @{PROPERTYName(value)}
 
-[PropertyNAME.lg]
+> TODO: Would be good to split out all the component pieces, i.e. help, cancel, confirmation into their own files
+[PROPERTYName.lg]
+[Confirmation.lg]

packages/dialog/templates/en-us/locale.lg.lg

@@ -2,4 +2,4 @@
 - @{formName}.@{locale}.lg
 
 # template
-- >>> Library\n@{join(foreach(templates.lg, lg, concat('[', lg.fullName, '](./', lg.fullName, ')')), '\n')}
+- >>> Library\n@{join(foreach(templates.lg, lg, concat('[', lg.fallbackName, '](', lg.fallbackName, ')')), '\n')}

packages/dialog/templates/enumAsk.dialog.lg

@@ -25,4 +25,4 @@
 ```
 
 # callAsk
-- @\{Ask{property}()}
+- @\{Ask@{property}()}