Added support for Packer 1.0 and W10 Creators Update

List of changes:

* gitignore updates to avoid polluting bin/ folder and adding packermerge/packages folder
* Answer file updated to win 1703
* Packer scripts updated to work with Packer 1.0
* Solved autounattend.xml issues in windows 10 (autologon)
* VMSConfigGen, generator of VMS section for vmgen.json
* Add Windows script to install the required applications for the build and execution of the whole project. Uses Chocolatey to install the apps.
* Shell script to install the required programs using Homebrew
* Update to vmware-windows10: Update vm hw to v12
* Files for win10 preview (build 16215)
* Fix various minor issues and edge cases with vms.json
* Attach vms.json to email notification
* Add script to rename the os version in the output file vms.json
* Add retry when the generation process fails one time.

Author: acasquete

.gitignore

@@ -4,7 +4,7 @@
 *.md5.txt
 *.zip.00?
 !scripts/**/*.exe
-!bin/**/*.exe
+bin/
 # Packer builder outputs
 outputs/
 output-hyperv-iso/
@@ -16,15 +16,18 @@ output-parallels-pvm/
 *.box
 generated/
 vms/output/
+scripts/template-output/
 
 *.user
 tools/VMSGen/VMSGen/obj/
 tools/VMSGen/VMSGen/bin/
 tools/VMSGen/packages/
+tools/PackerMerge/packages
 tools/PackerMerge/PackerMerge/bin/
 tools/PackerMerge/PackerMerge/obj/
 tools/PackerMerge/.vs/config/applicationhost.config
 tools/VMSGen/.vs/config/applicationhost.config
+tools/VMSConfigGen/VMSConfigGen/obj
 
 *.suo
 bin/*.vshost.exe.*
@@ -35,4 +38,5 @@ scripts/vmgen.json
 scripts/floppy/build.cfg
 scripts/vmgen.log
 scripts/vmgen.cfg
+scripts/parallels-command.txt
 *.user

README.md

@@ -12,7 +12,7 @@ Currently this process only works on Windows 8.1 machines.
 
 ## Software Requirements
 * Windows 8.1 (working on a solution that works for Windows 10)
-* [Packer 0.8.6 For Windows and Mac](https://packer.io/downloads.html) (It's an old version, we know)
+* [Packer 1.0.2 For Windows and Mac](https://packer.io/downloads.html) 
 * [VirtualBox 5.0.2 for Windows hosts](https://www.virtualbox.org/wiki/Downloads)
 * [VMware Workstation](http://www.vmware.com/products/workstation)
 * Hyper-V
@@ -22,6 +22,12 @@ Currently this process only works on Windows 8.1 machines.
 * [Putty](http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html)
 * [Visual Studio Community](https://www.visualstudio.com/downloads/)
 
+### Automatic installation in Windows computers
+For an automatic installation of the required software you can use the script `.\scripts\apps\winappinstaller.ps1` It uses Chocolatey for installing the programs in the previous list. It is not recommended to use the script in machines that have already some of the programs installed manually. The purpose of the script is saving time in the installation in clean machines.
+
+### Automatic installation in Mac computers
+Although in Mac only Packer and Parallels are required, you can install them automatically using the script `.\scripts\apps\macappinstaller.sh`. The script uses Homebrew to install the apps. Parallels will require in addition of the installation the registration of a valid key. In case packer is not installing correctly write in the terminal `brew install packer` to retry the installation.
+
 ## Preparing the Windows build environment
 
 The following are the instructions to set up an environment almost fully automated. During the VM creation process sometimes the host needs to be rebooted or some permissions are required. If you don't some manual intervention you can skip some steps like AutoLogon, etc.
@@ -118,6 +124,11 @@ Turn Apple File Sharing on
 ```
 sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.AppleFileServer.plist
 ```
+Activate SMB
+
+```
+sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.smbd.plist
+```
 
 Share Repository folder
 
@@ -126,10 +137,12 @@ sudo sharing -a /Users/admin/dev.microsoftedge.com-vms/
 ```
 
 ### Install Packer
+(Note: this step is not necessary if the script `.\scripts\apps\macappinstaller.sh` has been executed.)
 
 Unzip Packer files to /Users/packer
 
 ### Set environment variable on OS X
+(Note: this step is not necessary if the script `.\scripts\apps\macappinstaller.sh` has been executed.)
 
 * Open up Terminal.
 * Run the following command:
@@ -149,6 +162,8 @@ The Apple Mac OS X operating system has SSH installed by default but the SSH dae
 
 To enable it, go to "System Preferences". Under "Internet & Networking" there is a "Sharing" icon. Run that. In the list that appears, check the "Remote Login" option.
 
+**Important**: once SSH is enabled in the Mac, we have to connect to it manually by Putty SSH from the Windows computer prepared to initiate the process. This way the Mac is recognized by the Putty when the process is executed. If we do not register the Mac as an accepted connection, the SSH connection will show this error in the generation process `The server's host key is not cached in the registry. You have no guarantee that the server is the computer you think it is. The server's rsa2 key finger (...) Connection abandoned.`    
+
 ## Download Windows ISOs
 
 This process uses ISOs, and more precisely the Client Enterprise Evaluation. You should legally get a copy of the iso for the guest you want to build and put it directly in the the `scripts\iso` folder.
@@ -181,8 +196,7 @@ This script requires a configuration file called `vmgen.json` located in the sam
     "SSH_User": "admin",
     "SSH_Password": "password",
     "NetworkPath": "\\\\MAC\\microsoftedge-vms",
-    "RepoPath" : "/Users/admin/dev.microsoftedge.com-vms",
-    "PackerPath" : "/Users/admin/packer"
+    "RepoPath" : "/Users/admin/dev.microsoftedge.com-vms"
   },
   "Mail": {
     "SMTP": "smtp.office365.com",
@@ -191,6 +205,10 @@ This script requires a configuration file called `vmgen.json` located in the sam
     "User": "[email protected]",
     "Password": ""
   },
+  "OsRenaming": {
+    "Win10": "Win10 (x64) Build xxxx",
+    "Win81": "Win81 (x86) Build yyyy"
+	},
   "VMS": {
     "Windows": {
       "HyperV": {
@@ -233,6 +251,7 @@ This script requires a configuration file called `vmgen.json` located in the sam
 * AzureStorage - Contains the Url and the Key of de Azure Storage Account to upload de output files.
 * Mac - Contains IP, SSH User & password, and a shared path.
 * Mail - SMTP configuration to send emails to the appropriate people
+* OsRenaming (optional) - Indicates the final names desired in the output JSON for the os versions. Let's say that periodically we execute the process for the Win10 and we want in the output JSON the specific build version "Win10 (x64) build 2345". This renamed version of the output JSON is the one sent by mail. If a rename is not desired simply remove this section.
 * VMS - Object struct to set the dev.microsoftedge.com Virtual Machines to be generated. The valid values for each level are as follows:
 
   * **First Level** - Windows, Mac
@@ -251,6 +270,18 @@ To upload the generated files after a build without regenerate de VMs, we need t
 
 A JSON specific version will be generated in OutputPath. If you want your result to be merge with another file, place it in the same folder with the name vms.json.
 
+### Renaming the os version in the output JSON
+
+**Automatic renaming**: to rename the output JSON automatically in the process, just add the `OsRenaming` section in the vmgen file. When the generation process is executed, a copy of the output JSON with the renaming will be stored in the output path, inside the notification folder. This version is the one that will be sent by email to the configured mail receiver. 
+
+**Manual renaming**: if you want to rename the output JSON manually you can use the `scripts\vmsrename.ps1`. For instance if we want to change the value Win10 used in the generation process for an explicit name with build and architecture:  
+
+```
+.\vmsrename.ps1 "Win10" "Win10 (x86) Build 6307"
+```
+
+The input file `vms.json` must be in the same folder as the script, and the output file will be `vms_renamed.json`.
+
 ## Generate a new OS template
 
 To generate a new platform perform the following steps:

scripts/answer_files/10/autounattend.xml

@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 <unattend xmlns="urn:schemas-microsoft-com:unattend">
     <servicing />
     <settings pass="windowsPE">
@@ -45,8 +45,8 @@
                     <InstallToAvailablePartition>false</InstallToAvailablePartition>
                     <InstallFrom>
                         <MetaData wcm:action="add">
-                            <Key>/IMAGE/INDEX</Key>
-                            <Value>1</Value>
+                            <Key>/IMAGE/NAME</Key>
+                            <Value>Windows 10 Enterprise Evaluation</Value>
                         </MetaData>
                     </InstallFrom>
                 </OSImage>
@@ -64,7 +64,7 @@
         </component>
     </settings>
     <settings pass="offlineServicing">
-        <component name="Microsoft-Windows-LUA-Settings" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
+        <component name="Microsoft-Windows-LUA-Settings" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
             <EnableLUA>false</EnableLUA>
         </component>
     </settings>
@@ -94,14 +94,6 @@
                 <NetworkLocation>Home</NetworkLocation>
                 <ProtectYourPC>1</ProtectYourPC>
             </OOBE>
-            <AutoLogon>
-                <Password>
-                    <Value>UABhAHMAcwB3ADAAcgBkACEAUABhAHMAcwB3AG8AcgBkAA==</Value>
-                    <PlainText>false</PlainText>
-                </Password>
-                <Username>IEUser</Username>
-                <Enabled>true</Enabled>
-            </AutoLogon>
             <FirstLogonCommands>
                 <SynchronousCommand wcm:action="add">
                     <CommandLine>cmd.exe /c powershell -Command "Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Force"</CommandLine>
@@ -151,23 +143,58 @@
                     <Description>Disable password expiration for IEUser user</Description>
                 </SynchronousCommand>
                 <SynchronousCommand wcm:action="add">
-                    <CommandLine>REG ADD &quot;HKLM\System\CurrentControlSet\Control\Network\NewNetworkWindowOff&quot;</CommandLine>
+                    <CommandLine>REG ADD "HKLM\System\CurrentControlSet\Control\Network\NewNetworkWindowOff"</CommandLine>
                     <Description>Disable set Network Location</Description>
                     <Order>12</Order>
                     <RequiresUserInput>true</RequiresUserInput>
                 </SynchronousCommand>
                 <SynchronousCommand wcm:action="add">
-                    <CommandLine> REG ADD “HKLM\System\CurrentControlSet\Services\Netlogon\Parameters” /v DisablePasswordChange /t REG_DWORD /d 2 /f</CommandLine>
+                    <CommandLine> REG ADD HKLM\System\CurrentControlSet\Services\Netlogon\Parameters /v DisablePasswordChange /t REG_DWORD /d 2 /f</CommandLine>
                     <Description>Disable computer password change</Description>
                     <Order>13</Order>
                     <RequiresUserInput>true</RequiresUserInput>
                 </SynchronousCommand>
+                <SynchronousCommand wcm:action="add">
+                    <CommandLine>REG ADD HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System /v EnableLUA /t REG_DWORD /d 0 /f</CommandLine>
+                    <Description>Disable UAC</Description>
+                    <RequiresUserInput>true</RequiresUserInput>
+                    <Order>14</Order>
+                </SynchronousCommand>
+                <SynchronousCommand wcm:action="add">
+                    <CommandLine>cmd.exe /c powershell -File a:\setautologon.ps1</CommandLine>
+                    <Description>Autologon</Description>
+                    <RequiresUserInput>true</RequiresUserInput>
+                    <Order>15</Order>
+                </SynchronousCommand>                
                 <SynchronousCommand wcm:action="add">
                     <CommandLine>wscript a:\install.vbs</CommandLine>
                     <Description>Change background with System Information</Description>
                     <RequiresUserInput>true</RequiresUserInput>
-                    <Order>14</Order>
+                    <Order>16</Order>
+                </SynchronousCommand>
+                <!-- Autologon entries for 1703-->
+                <!--
+                <SynchronousCommand wcm:action="add">
+                    <CommandLine> REG ADD "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /v "AutoAdminLogon" /t REG_SZ /d "1" /f</CommandLine>
+                    <Description>Enable autologon (fix for 1703) 1/4</Description>
+                    <RequiresUserInput>true</RequiresUserInput>
+                    <Order>80</Order>
                 </SynchronousCommand>
+                <SynchronousCommand wcm:action="add">
+                    <CommandLine>reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /v "DefaultPassword" /t REG_SZ /d "Passw0rd!" /f</CommandLine>
+                    <Description>Enable autologon (fix for 1703) 2/4</Description>
+                    <RequiresUserInput>true</RequiresUserInput>
+                    <Order>81</Order>
+                </SynchronousCommand>             
+                <SynchronousCommand wcm:action="add">
+                    <CommandLine>reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /v "AutoAdminLogonCount" /t REG_DWORD /d 10 /f</CommandLine>
+                    <Description>Enable autologon (fix for 1703) 4/4</Description>
+                    <RequiresUserInput>true</RequiresUserInput>
+                    <Order>83</Order>
+                </SynchronousCommand> 
+                -->
+
+                <!-- End autologon -->
                 <SynchronousCommand wcm:action="add">
                     <CommandLine>cmd.exe /c a:\microsoft-updates.bat</CommandLine>
                     <Order>98</Order>
@@ -181,6 +208,22 @@
                 </SynchronousCommand>
             </FirstLogonCommands>
             <ShowWindowsLive>false</ShowWindowsLive>
+            <AutoLogon>
+                <Password>
+                    <Value>UABhAHMAcwB3ADAAcgBkACEAUABhAHMAcwB3AG8AcgBkAA==</Value>
+                    <PlainText>false</PlainText>
+                </Password>
+                <LogonCount>10</LogonCount>
+                <Enabled>true</Enabled>
+                <Username>IEUser</Username>
+            </AutoLogon>
+        </component>
+        <component name="Microsoft-Windows-International-Core" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+            <InputLocale>en-US</InputLocale>
+            <SystemLocale>en-US</SystemLocale>
+            <UILanguage>en-US</UILanguage>
+            <UILanguageFallback>en-US</UILanguageFallback>
+            <UserLocale>en-US</UserLocale>
         </component>
     </settings>
     <settings pass="specialize">
@@ -197,4 +240,5 @@
             <SkipAutoActivation>true</SkipAutoActivation>
         </component>
     </settings>
+    <cpi:offlineImage cpi:source="wim:d:/iso/sources/install.wim#Windows 10 Enterprise Evaluation" xmlns:cpi="urn:schemas-microsoft-com:cpi" />
 </unattend>