Feature/text field precision (#35)

jdschleicher · web-flow · commit d2d41939f85c · 2025-11-24T05:26:39.000-05:00
* feat: add methods for building text and numeric recipe values with constraints

- Added buildTextRecipeValueWithLength(length: number) and buildNumericRecipeValueWithPrecision(precision: number) to IRecipeFakerService interface
- Implemented methods in FakerJSRecipeFakerService and SnowfakeryRecipeFakerService to generate faker data respecting length/precision limits
- Updated RecipeService to utilize new methods for improved fake data generation in text and numeric fields

* feat(recipe-faker): add scale support to numeric recipe value generation

- Renamed `buildNumericRecipeValueWithPrecision` to `buildNumericRecipeValueWithPrecisionAndScale` in IRecipeFakerService and implementations
- Added optional `scale` parameter to handle decimal fields (currency, percent) with appropriate precision and decimal places
- Updated FakerJS and Snowfakery services to generate integers for scale=0 and decimals for scale&gt;0
- Modified RecipeService to pass scale for percent fields, ensuring accurate fake data generation for numeric types

* fix(faker): cap numeric precision at 15 to avoid JavaScript number overflow

Updated FakerJSRecipeFakerService and SnowfakeryRecipeFakerService to limit precision to 15 digits for integer and decimal fields, preventing loss of accuracy in large numbers. Adjusted test expectations in FakerJSRecipeService.test.ts to reflect the new max value (10^15 - 1) for integer fields.

* refactor: simplify numeric value generation and add currency-specific method

- Refactored buildNumericRecipeValueWithPrecisionAndScale in FakerJS and Snowfakery services to remove redundant calculations and improve readability.
- Added buildCurrencyRecipeValueWithPrecisionAndScale method to handle currency fields separately, ensuring full precision usage.
- Updated IRecipeFakerService interface to include the new currency method.

* feat: format faker expressions as YAML block scalars in recipes

Update buildPercentRecipeValueWithPrecisionAndScale and buildCurrencyRecipeValueWithPrecisionAndScale to prepend `|` and add indentation to the returned strings, ensuring they are valid YAML literal block scalars. Adjust corresponding test expectations to match the new formatting for accurate assertion in FakerJS recipe service tests. This improves YAML recipe generation by preserving multi-line structure in generated fake values.

* fix: correct max value calculation for numeric and currency fields

Refactor the buildNumericRecipeValueWithPrecisionAndScale and buildCurrencyRecipeValueWithPrecisionAndScale methods to use precision minus scale as the digit count left of the decimal for max value generation, fixing inaccurate faker outputs for decimal fields. Updated corresponding test expectations.

* fix(RecipeFakerService): correctly calculate left_digits for numeric and currency fields based on precision - scale

- Updated buildNumericRecipeValueWithPrecisionAndScale and buildCurrencyRecipeValueWithPrecisionAndScale to use left_digits = precision - effectiveScale, ensuring accurate representation of digits before the decimal point.
- Adjusted the corresponding test to expect left_digits=16 for a currency field with 18 precision and 2 scale, matching the correct calculation. This fixes a potential issue where total precision was mistakenly used as left_digits, leading to incorrect fake data generation.

* fix: add missing closing braces and remove comments in SnowfakeryRecipeFakerService

Removed inline comments from buildNumericRecipeValueWithPrecisionAndScale and
buildCurrencyRecipeValueWithPrecisionAndScale methods, and added missing closing
braces to fix syntax errors.

* feat: Enhance numeric field precision and fix related bugs

- Add support for scale in numeric value generation for accurate currency and number fields
- Format faker expressions as YAML block scalars and add constraints for text/numeric values
- Fix max value calculations, precision-based left digits, JS overflow, and code cleanup
- Bump version to 2.7.0

* docs: clean up changelog by removing preliminary unreleased features and fixes

Remove entries for numeric scale support, YAML formatting, value constraints, and bug fixes in numeric handling to reflect accurate pre-release state, preventing confusion in release notes.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,46 @@
 # Change Log
 
+## [2.7.0] [PR#35](https://github.com/jdschleicher/Salesforce-Data-Treecipe/pull/35) - Feature: Enhanced Text & Numeric Field Precision Handling
+
+### 🎯 Major Features
+
+#### 1. **Text and Numeric Value Constraints**
+Added new methods for building text and numeric recipe values with constraints, enhancing control over generated data ranges and formats.
+
+### 🔧 Technical Details
+
+**Code Example - Currency Field XML to FakerJS YAML**:
+
+Given a custom object field XML markup with precision and scale:
+
+```xml
+<fields>
+  <fullName>Price__c</fullName>
+  <description>Product price</description>
+  <externalId>false</externalId>
+  <label>Price</label>
+  <precision>8</precision>
+  <required>false</required>
+  <scale>2</scale>
+  <type>Currency</type>
+  <unique>false</unique>
+</fields>
+```
+
+The generated recipe automatically creates a faker expression that respects the precision (8) and scale (2):
+
+```yaml
+- object: My_Custom_Object__c
+  fields:
+    Price__c: ${{ faker.finance.amount({ min: 0, max: 999999, dec: 2 }) }}
+```
+
+This ensures the generated currency values:
+- Have at most 8 total digits (precision)
+- Have exactly 2 decimal places (scale)
+- Are within the valid range (0 to 99,999.99)
+- Match Salesforce field constraints
+
 ## [2.6.0] [PR#34](https://github.com/jdschleicher/Salesforce-Data-Treecipe/pull/34) - Feature: Relationship Service & Bug Fix for Special Characters in Picklists
 
 ### 🎯 Major Features
diff --git a/package.json b/package.json
@@ -3,7 +3,7 @@
   "displayName": "Salesforce Data Treecipe",
   "description": "source-fidelity driven development, pairs well with cumulus-ci",
   "icon": "images/datatreecipe.webp",
-  "version": "2.6.0",
+  "version": "2.7.0",
   "engines": {
     "vscode": "^1.94.0"
   },
diff --git a/src/treecipe/src/RecipeFakerService.ts/FakerJSRecipeFakerService/FakerJSRecipeFakerService.ts b/src/treecipe/src/RecipeFakerService.ts/FakerJSRecipeFakerService/FakerJSRecipeFakerService.ts
@@ -504,7 +504,37 @@ ${this.generateTabs(5)}${randomChoicesBreakdown}`;
     getMultipicklistTODOPlaceholderWithExample():string {
 
         const emptyMultiSelectXMLDetailPlaceholder = `### TODO: POSSIBLE GLOBAL OR STANDARD VALUE SET USED FOR THIS MULTIPICKLIST AS DETAILS ARE NOT IN FIELD XML MARKUP -- FIND ASSOCIATED VALUE SET AND REPLACE COMMA SEPARATED FRUITS WITH VALUE SET OPTIONS: \${{ (faker.helpers.arrayElements(['apple', 'orange', 'banana']) ).join(';') }}`;
-        return emptyMultiSelectXMLDetailPlaceholder;    
+        return emptyMultiSelectXMLDetailPlaceholder;
+
+    }
+
+    buildTextRecipeValueWithLength(length: number): string {
+        return `${this.openingRecipeSyntax} faker.lorem.text(${length}).substring(0, ${length}) ${this.closingRecipeSyntax}`;
+    }
+
+    buildNumericRecipeValueWithPrecisionAndScale(precision: number, scale?: number): string {
+
+        const effectiveScale = scale ?? 0;
+        const maxNumbersLeftOfDecimal = '9'.repeat(precision - effectiveScale);
+
+        if (effectiveScale === 0) {
+            return `|
+            ${this.openingRecipeSyntax} faker.number.int({min: 0, max: ${maxNumbersLeftOfDecimal}}) ${this.closingRecipeSyntax}`;
+        } else {
+            return `|
+            ${this.openingRecipeSyntax} faker.finance.amount({min: 0, max: ${maxNumbersLeftOfDecimal}, dec: ${effectiveScale}}) ${this.closingRecipeSyntax}`;
+        }
+
+    }
+
+    buildCurrencyRecipeValueWithPrecisionAndScale(precision: number, scale?: number): string {
+        // Special handling for currency fields - use full precision as left_digits
+
+        const effectiveScale = scale ?? 0;
+        const maxNumbersLeftOfDecimal = '9'.repeat(precision - effectiveScale);
+
+        return `|
+            ${this.openingRecipeSyntax} faker.finance.amount({min: 0, max: ${maxNumbersLeftOfDecimal}, dec: ${effectiveScale}}) ${this.closingRecipeSyntax}`;
 
     }
 
diff --git a/src/treecipe/src/RecipeFakerService.ts/IRecipeFakerService.ts b/src/treecipe/src/RecipeFakerService.ts/IRecipeFakerService.ts
@@ -28,4 +28,7 @@ export interface IRecipeFakerService {
         controllingValue: string): string
     getMultipicklistTODOPlaceholderWithExample(): string
     getStandardAndGlobalValueSetTODOPlaceholderWithExample(): string
-}
+    buildTextRecipeValueWithLength(length: number): string
+    buildNumericRecipeValueWithPrecisionAndScale(precision: number, scale?: number): string
+    buildCurrencyRecipeValueWithPrecisionAndScale(precision: number, scale?: number): string
+}
diff --git a/src/treecipe/src/RecipeFakerService.ts/SnowfakeryRecipeFakerService/SnowfakeryRecipeFakerService.ts b/src/treecipe/src/RecipeFakerService.ts/SnowfakeryRecipeFakerService/SnowfakeryRecipeFakerService.ts
@@ -397,7 +397,32 @@ ${this.generateTabs(5)}${randomChoicesBreakdown}`;
 
         const emptyMultiSelectXMLDetailPlaceholder = `### TODO: POSSIBLE GLOBAL OR STANDARD VALUE SET USED FOR THIS MULTIPICKLIST AS DETAILS ARE NOT IN FIELD XML MARKUP -- FIND ASSOCIATED VALUE SET AND REPLACE COMMA SEPARATED FRUITS WITH VALUE SET OPTIONS: \${{ (';').join((fake.random_sample(elements=('apple', 'orange', 'banana')))) }}`;
         return emptyMultiSelectXMLDetailPlaceholder;
+
+    }
+
+    buildTextRecipeValueWithLength(length: number): string {
+        return `${this.openingRecipeSyntax}fake.text(max_nb_chars=${length})${this.closingRecipeSyntax}`;
+    }
+
+    buildNumericRecipeValueWithPrecisionAndScale(precision: number, scale?: number): string {
+
+        const effectiveScale = scale ?? 0;
+        const maxNumbersLeftOfDecimal = '9'.repeat(precision - effectiveScale);
+
+        if (effectiveScale === 0) {
+            return `${this.openingRecipeSyntax}fake.random_int(min=0, max=${maxNumbersLeftOfDecimal})${this.closingRecipeSyntax}`;
+        } else {
+            return `${this.openingRecipeSyntax}fake.pydecimal(left_digits=${maxNumbersLeftOfDecimal}, right_digits=${effectiveScale}, positive=True)${this.closingRecipeSyntax}`;
+        }
+    
+    }
+
+    buildCurrencyRecipeValueWithPrecisionAndScale(precision: number, scale?: number): string {
         
+        const effectiveScale = scale ?? 0;
+        const maxNumbersLeftOfDecimal = precision - effectiveScale;
+        return `${this.openingRecipeSyntax}fake.pydecimal(left_digits=${maxNumbersLeftOfDecimal}, right_digits=${effectiveScale}, positive=True)${this.closingRecipeSyntax}`;
+    
     }
 
-}
+}
diff --git a/src/treecipe/src/RecipeService/RecipeService.ts b/src/treecipe/src/RecipeService/RecipeService.ts
@@ -95,32 +95,50 @@ export class RecipeService {
                     return fakeRecipeValue;
                     
                 case 'multiselectpicklist':
-    
+
                     if ( !(xmlFieldDetail.picklistValues) ) {
                         // THIS SCENARIO INDICATEDS THAT THE PICKLIST FIELD UTILIZED A GLOBAL VALUE SET
                         const emptyMultiSelectXMLDetailPlaceholder = this.fakerService.getMultipicklistTODOPlaceholderWithExample();
                         return emptyMultiSelectXMLDetailPlaceholder;
                     }
                     const availablePicklistChoices = xmlFieldDetail.picklistValues.map(picklistOption => picklistOption.picklistOptionApiName);
-                    fakeRecipeValue = this.fakerService.buildMultiSelectPicklistRecipeValueByXMLFieldDetail(availablePicklistChoices, 
+                    fakeRecipeValue = this.fakerService.buildMultiSelectPicklistRecipeValueByXMLFieldDetail(availablePicklistChoices,
                                                                                                             recordTypeApiToRecordTypeWrapperMap,
                                                                                                             xmlFieldDetail.apiName
                                                                                                         );
-            
+
                     return fakeRecipeValue;
-                    
-                // case 'masterdetail':
-                    
-                //     return {
-                //         type: 'lookup'
-                //     };
-    
-                // case 'lookup':
-    
-                //     return 'test';
-                    
-                default: 
-    
+
+                case 'text':
+                case 'textarea':
+                case 'longtextarea':
+                case 'html':
+
+                    if (xmlFieldDetail.length) {
+                        fakeRecipeValue = this.fakerService.buildTextRecipeValueWithLength(xmlFieldDetail.length);
+                        return fakeRecipeValue;
+                    }
+                    // Fall through to default if no length
+
+                case 'number':
+                case 'percent':
+
+                    if (xmlFieldDetail.precision) {
+                        fakeRecipeValue = this.fakerService.buildNumericRecipeValueWithPrecisionAndScale(xmlFieldDetail.precision, xmlFieldDetail.scale);
+                        return fakeRecipeValue;
+                    }
+                    // Fall through to default if no precision
+
+                case 'currency':
+
+                    if (xmlFieldDetail.precision) {
+                        fakeRecipeValue = this.fakerService.buildCurrencyRecipeValueWithPrecisionAndScale(xmlFieldDetail.precision, xmlFieldDetail.scale);
+                        return fakeRecipeValue;
+                    }
+                    // Fall through to default if no precision
+
+                default:
+
                     fakeRecipeValue = this.getFakeValueIfExpectedSalesforceFieldType(fieldType);
                     return fakeRecipeValue;
     
@@ -296,4 +314,4 @@ ${this.generateTabs(1)}${fieldPropertAndRecipeValue}`;
 
     }
 
-}
+}
diff --git a/src/treecipe/src/RecipeService/tests/FakerJSRecipeService.test.ts b/src/treecipe/src/RecipeService/tests/FakerJSRecipeService.test.ts
@@ -191,9 +191,9 @@ describe('FakerJSRecipeService IRecipeService Implementation Shared Intstance Te
         test('given expected number XMLFieldDetail, returns the expected fakerJS YAML recipe value', () => {
 
             const expectedXMLDetailForNumber:XMLFieldDetail = XMLMarkupMockService.getNumberXMLFieldDetail();
-            const expectedFakerJSExpressionForNumber =  `|
-                \${{ faker.number.int({min: 0, max: 999999}) }}`;
-            const recordTypeNameByRecordTypeNameToXMLMarkup = {};   
+            const expectedFakerJSExpressionForNumber = `|
+            \${{ faker.number.int({min: 0, max: 999999999999999999}) }}`;
+            const recordTypeNameByRecordTypeNameToXMLMarkup = {};
 
             const actualFakerJSForNumber = recipeServiceWithFakerJS.getRecipeFakeValueByXMLFieldDetail(expectedXMLDetailForNumber, recordTypeNameByRecordTypeNameToXMLMarkup);
             expect(actualFakerJSForNumber).toBe(expectedFakerJSExpressionForNumber);
@@ -203,7 +203,8 @@ describe('FakerJSRecipeService IRecipeService Implementation Shared Intstance Te
         test('given expected currency XMLFieldDetail, returns the expected fakerJS YAML recipe value', () => {
 
             const expectedXMLDetailForCurrency:XMLFieldDetail = XMLMarkupMockService.getCurrencyFieldDetail();
-            const expectedFakerJSExpressionForCurrency = "\${{ faker.finance.amount(0, 999999, 2) }}";
+            const expectedFakerJSExpressionForCurrency = `|
+            \${{ faker.finance.amount({min: 0, max: 9999999999999999, dec: 2}) }}`;
             const recordTypeNameByRecordTypeNameToXMLMarkup = {};
             const actualFakerJSForCurrency = recipeServiceWithFakerJS.getRecipeFakeValueByXMLFieldDetail(expectedXMLDetailForCurrency, recordTypeNameByRecordTypeNameToXMLMarkup);
 
diff --git a/src/treecipe/src/RecipeService/tests/RecipeService.test.ts b/src/treecipe/src/RecipeService/tests/RecipeService.test.ts
@@ -124,8 +124,8 @@ describe('SnowfakeryRecipeService IRecipeService Implementation Shared Intstance
         test('given expected number XMLFieldDetail, returns the expected snowfakery YAML recipe value', () => {
 
             const expectedXMLDetailForNumber:XMLFieldDetail = XMLMarkupMockService.getNumberXMLFieldDetail();
-            const expectedSnowfakeryValueForNumber = '${{fake.random_int(min=0, max=999999)}}';
-            const recordTypeNameByRecordTypeNameToXMLMarkup = {};   
+            const expectedSnowfakeryValueForNumber = '${{fake.random_int(min=0, max=999999999999999999)}}';
+            const recordTypeNameByRecordTypeNameToXMLMarkup = {};
 
             const actualSnowfakeryValueForNumber = recipeServiceWithSnow.getRecipeFakeValueByXMLFieldDetail(expectedXMLDetailForNumber, recordTypeNameByRecordTypeNameToXMLMarkup);
 
@@ -136,14 +136,25 @@ describe('SnowfakeryRecipeService IRecipeService Implementation Shared Intstance
         test('given expected currency XMLFieldDetail, returns the expected snowfakery YAML recipe value', () => {
 
             const expectedXMLDetailForCurrency:XMLFieldDetail = XMLMarkupMockService.getCurrencyFieldDetail();
-            const expectedSnowfakeryValueForCurrency = '${{fake.pydecimal(left_digits=6, right_digits=2, positive=True)}}';
+            const expectedSnowfakeryValueForCurrency = '${{fake.pydecimal(left_digits=16, right_digits=2, positive=True)}}';
             const recordTypeNameByRecordTypeNameToXMLMarkup = {};
             const actualSnowfakeryValueForCurrency = recipeServiceWithSnow.getRecipeFakeValueByXMLFieldDetail(expectedXMLDetailForCurrency, recordTypeNameByRecordTypeNameToXMLMarkup);
 
             expect(actualSnowfakeryValueForCurrency).toBe(expectedSnowfakeryValueForCurrency);
 
         });
 
+        test('given expected text XMLFieldDetail with length, returns the expected snowfakery YAML recipe value with length limit', () => {
+
+            const expectedXMLDetailForTextWithLength:XMLFieldDetail = XMLMarkupMockService.getTextXMLFieldDetailWithLength();
+            const expectedSnowfakeryValueForTextWithLength = '${{fake.text(max_nb_chars=50)}}';
+            const recordTypeNameByRecordTypeNameToXMLMarkup = {};
+            const actualSnowfakeryValueForTextWithLength = recipeServiceWithSnow.getRecipeFakeValueByXMLFieldDetail(expectedXMLDetailForTextWithLength, recordTypeNameByRecordTypeNameToXMLMarkup);
+
+            expect(actualSnowfakeryValueForTextWithLength).toBe(expectedSnowfakeryValueForTextWithLength);
+
+        });
+
     });
 
     describe('initiateRecipeByObjectName', () => {
@@ -407,4 +418,3 @@ describe('SnowfakeryRecipeService IRecipeService Implementation Shared Intstance
     });
 
 });
-
diff --git a/src/treecipe/src/XMLProcessingService/XMLFieldDetail.ts b/src/treecipe/src/XMLProcessingService/XMLFieldDetail.ts
@@ -10,4 +10,7 @@ export class XMLFieldDetail {
     public controllingField?: string;
     public xmlMarkup: string;
     public isStandardValueSet?: boolean;
-}
+    public precision?: number;
+    public scale?: number;
+    public length?: number;
+}
diff --git a/src/treecipe/src/XMLProcessingService/XmlFileProcessor.ts b/src/treecipe/src/XMLProcessingService/XmlFileProcessor.ts
@@ -37,6 +37,22 @@ export class XmlFileProcessor {
       xmlFieldDetail.fieldType = typeValue;
       xmlFieldDetail.fieldLabel = fieldLabel;
 
+      // Parse precision, scale, and length properties (only if they exist in XML)
+      const precision = fieldXML?.CustomField?.precision?.[0];
+      if (precision !== undefined && precision !== null) {
+        xmlFieldDetail.precision = parseInt(precision, 10);
+      }
+
+      const scale = fieldXML?.CustomField?.scale?.[0];
+      if (scale !== undefined && scale !== null) {
+        xmlFieldDetail.scale = parseInt(scale, 10);
+      }
+
+      const length = fieldXML?.CustomField?.length?.[0];
+      if (length !== undefined && length !== null) {
+        xmlFieldDetail.length = parseInt(length, 10);
+      }
+
       if ( typeValue === 'Picklist' || typeValue === "MultiselectPicklist") {
 
         let picklistValueSetMarkup = fieldXML.CustomField.valueSet?.[0];
@@ -215,8 +231,3 @@ export class XmlFileProcessor {
 
 
 }
-
-  
-
-
-
diff --git a/src/treecipe/src/XMLProcessingService/tests/XmlFileProcessor.test.ts b/src/treecipe/src/XMLProcessingService/tests/XmlFileProcessor.test.ts
diff --git a/src/treecipe/src/XMLProcessingService/tests/mocks/XMLMarkupMockService.ts b/src/treecipe/src/XMLProcessingService/tests/mocks/XMLMarkupMockService.ts
