diff --git a/astro.config.mjs b/astro.config.mjs index a385040..e1fee8b 100644 --- a/astro.config.mjs +++ b/astro.config.mjs @@ -114,7 +114,7 @@ export default defineConfig({ }, RomainDegrave: { name: "Romain Degrave", - title: "Master degree intern in software analysis", + title: "PHD student", url: "https://github.com/RomainDeg", picture: "https://avatars.githubusercontent.com/u/98580879" }, diff --git a/src/content/docs/blog/2025-10-15-transformation-operations.mdx b/src/content/docs/blog/2025-10-15-transformation-operations.mdx new file mode 100644 index 0000000..f5ea23a --- /dev/null +++ b/src/content/docs/blog/2025-10-15-transformation-operations.mdx @@ -0,0 +1,294 @@ +--- +title: "Transformation operations : tooling for code transformation tools with seamless edits" +subtitle: >- + How to use transformation operations to apply code transformations to source files +date: 2025-10-15 +background: '/img/posts/bg-posts.jpg' +authors: +- RomainDegrave +comments: true +draft: true +tags: +- transformation +--- + +import OperationsSVG from './img/posts/2025-10-15-transformation-operations/uml_blogpost.drawio.svg'; + +Automated code transformation can be a tricky thing. +Not only must we ensure, of course, that our edits do not break the code and are applied to every code entity which needs such an edit, but we must also think about their integration in the codebase. +The code we manipulate is going to be the code that we and our collaborators have to understand, maintain and update on a daily basis. +It is thus mandatory that the results of our tooling solutions can be seamlessly integrated to the input codebase. + +This blog post will present a framework to build code transformations tools which can do just that, using transformation operations. + +## Another solution for code transformation? Why? + +If you have an interest in (semi-)automated code transformation, then you might have already read the series of three blog posts discussing this very subject. +The third one shows an issue in the presented tooling system, due to the use of the FAST visitor generating source code matching the visited FAST. + +This visitor introduces two unnecessary changes to the source code when generating a FAST: + +- **Missing comments:** because the comments are not represented in FAST, they are not re-generated and are lost after applying a transformation. +- **Formatting heterogeneity:** because the formatting of the code is made by the visitor, it might not be similar to the formatting used in the rest of the source code. + +The loss of comments is an obvious problem which does not need explaining, but the changes in formatting are also quite problematic. +Especially in long methods (which is not the greatest practice originally, but anyway :smile:), this means removing every carriage return included for readability. +For invoked methods with many parameters which could have been written on several lines, they will be rewritten on one line only, etc. +Overall, leaving the code identical except for the changes needed to complete the transformation task is a much safer outcome, and it was the motivation behind the implementation of the tools presented in this blog post. + +Another motivation was the creation of a simpler API to describe transformation logic. +As you will see in the rest of this post, the transformation operations do not involve making changes to the FAST on which you want to apply transformations, contrary to the tools described in the series of blog post on code transformation. + +## Transformation tools repository + +The transformation operations presented in this blog post are located on the same repository that contains the other tools implemented to be used in code transformation. To import them, use this baseline: + +```smalltalk +Metacello new + baseline: 'FASTJavaTools'; + repository: 'github://moosetechnology/FAST-Java-Tools:main/src'; + load +``` + +## Operations overview + +Before going through a small case study to see this system in action, let's have a quick look at the operations available and their API. + + + +This might look like a lot of information at first glance, but the system is rather simple and straightforward. +You can see a certain number of classes, each defining one type of operation to apply on a FAST tree, like deletion, replacement and insertion with different subclasses for each specific case of insertion. +Each operation can take two FASTJavaEntity as variables defined in the parent entity of every operation, the `AbstractTransformationOperation` class: + +- a node on which to apply the transformation, called the `anchorNode` +- another node this time created by the user to be inserted in some way in the entity, called the `transformedNode`. + +To use the operations, the setters for those two variables are going to be your best friends. +You can see similar setters in the subclasses, but with different names adapted to the operation context. +Those are just syntactic sugar, also setters for the `anchorNode` or the `transformedNode`. + +Additional variables are needed for insertion operations, like: + +- `selector`, taking a symbol representing the setter to use to insert your `transformedNode` into the `anchorNode`, or +- `insertBefore`, a boolean to decide if the inserted statement or declaration will be inserted before or after the `anchorNode` selected. + +Overall, that's all you need to know! + +## Transformation scenario + +:::note[Following along] +From this point forward, we will code a tool to showcase the use of transformation operations. +If you want to be able to test what is presented in this post, you may start by creating a new image and import the repository given previously. From there, either implement the tool step by step using the provided code snippets or browse the complete tool available on this [repository](https://github.com/RomainDeg/Moose-BlogPost-Transformation). +::: + +Our case study is going to be very simple, as we are simply going to create transformation operations on a small mock method created for the occasion, so no model creation or import would be needed. + +Here is the code of this mock method : + +```java +public int methodToTransform(String strParam) { + HelperClass receiver = new HelperClass(); + int callResult = receiver.doSomething(strParam); + int secondCallResult = receiver.getConstant(CONSTANT_ID); + System.out.println("I''m testing something here!!!"); + return callResult + secondCallResult; +} +``` + +Our first step is going to be creating a class, `TransformationOperationsShowcaseTool`, with a `sourceCode` method returning this method above, and a `generateFAST` method parsing it and returning the `FASTJavaMethodEntity` created : + +```smalltalk +generateFAST + + | tool | + tool := JavaSmaCCProgramNodeImporterVisitor new. + ^ (tool parseCodeMethodString: self sourceCode) + allFASTJavaMethodEntity first +``` + +This is all we need to get started! +Let's now get into the heart of the subject: defining our operations of code transformation. + +## Defining operations + +As the UML diagram showed us previously, defining a transformation operation is as simple as finding the suitable `anchorNode` and creating the necessary `transformedNode`. +We will see this in the following examples. + +### Statement deletion + +First thing on the agenda is a simple statement deletion, as it seems an air-headed developer forgot to remove a debug print in our method (couldn't be me!). +If I call that a "simple" operation, it is because in the case of a removal operation we only need to find the `anchorNode` which will be the FAST node to delete. + +Here's how it looks in our code : + +```smalltalk +nodeToDeleteIn: fast + + "System.out.println(""I'm testing something here!!!"");" + ^ fast statements at: 4 +``` + +```smalltalk +createDeleteOperationFor: fast + + | deleteOp | + deleteOp := DeleteOperation new. + deleteOp anchorNode: (self nodeToDeleteIn: fast). + ^ deleteOp +``` + +As you can see, fetching the information is pretty straightforward, thanks to our limited scenario in this mock method. +Defining our operation once we have the necessary information is just as easy however, simply setting this node as the `anchorNode`. + +### Parameter insertion + +Now that we've warmed up with this first try, let's go once step above, and add a parameter to our mock method. +For example, if in the fourth line (`int secondCallResult = receiver.getConstant(CONSTANT_ID);`) we'd like to use any identifier we'd like! + +One way to implement that is : + +```smalltalk +nodeToInsert + + | newNode variable | + variable := FASTJavaVariableExpression new. + variable name: 'constantID'. + + newNode := FASTJavaParameter new. + newNode + variable: variable; + type: FASTJavaIntTypeExpression new. + ^ newNode +``` + +```smalltalk +createSignatureInsertionOperationFor: fast + + | signInserOp | + signInserOp := SignatureInsertionOperation new. + signInserOp + anchorNode: fast; + transformedNode: self nodeToInsert; + selector: #addParameter:. + ^ signInserOp +``` + +This time, the `anchorNode` couldn't be easier to fetch, as it is the FAST node representing our method, i.e. the root of our FAST tree. +Creating the parameter node takes a few steps, but remember that you can use the `FASTDump` view of any existing node in an inspector to see how to create similar nodes if you have doubts on how the structure goes! :smile: +Defining the operation remains simple once we have fetched and created the necessary nodes, the only new step to think about in the case of an insertion is the `selector`. +As we have seen in the UML, this message takes a symbol representing the setter to use to insert your `transformedNode` into the `anchorNode`. +In our case, we add a parameter here. + +Finally, let's add one more transformation, so that our parameter is actually used in the code of the method! +It will replace the parameter given in the call to `getConstant`, in the fourth line : + +```smalltalk +nodeToReplaceIn: fast + + | originalNode | + "CONSTANT_ID" + originalNode := (fast statements at: 3) declarators first expression + arguments first. + ^ originalNode +``` + +```smalltalk +replacingNode + + | newNode | + newNode := FASTJavaVariableExpression new. + newNode name: 'constantID'. + ^ newNode +``` + +```smalltalk +createReplacingOperationFor: fast + + | replaceOp | + replaceOp := ReplacingOperation new. + replaceOp + originalNode: (self nodeToReplaceIn: fast); + transformedNode: self replacingNode. + ^ replaceOp +``` + +This concludes the part about operations definition. We will now see how to use those to actually transform the code of our method since for now, nothing has been actually transformed! + +## Using our operations to transform the source code + +This step is actually as simple as using a constructor and calling a method! :smile: + +We will now use a new tool, the class `FASTJavaCodeGeneratorKeepingFormatting`. +This class contains all the behavior necessary to take those operations and apply all necessary transformations so that the source code is modified exactly as defined in your operations. + +```smalltalk +setupCompleteTransformation + + | fast operations | + fast := self generateFAST. + + operations := OrderedCollection new. + operations + add: (self createDeleteOperationFor: fast); + add: (self createSignatureInsertionOperationFor: fast); + add: (self createReplacingOperationFor: fast). + + ^ FASTJavaCodeGeneratorKeepingFormatting + onSourceCode: self sourceCode + forOperations: operations +``` + +As you can see, this tool doesn't need a Famix or FAST entity on which to apply the operations, only the source code. +Since the `anchorNode` attributes of each operation hold `startPos` and `endPos` attributes that are consistent with the source code given by the FAST entity, no more information is needed. +Setting up the tool is the first step, now all that remains is using it! + +```smalltalk +applyAndViewTransformation + + | tool transformedCode | + tool := self setupCompleteTransformation. + transformedCode := tool applyCodeTransformation. + + ^ (TransformationEditor + openForOriginalText: self sourceCode + transformed: transformedCode + removedHighlights: tool removedHighlights + addedHighlights: tool addedHighlights + originalEntity: nil) fetchEditedCode +``` + +!["Transformation Editor View"](./img/posts/2025-10-15-transformation-operations/transformation_editor.PNG) + +Once the tool is set up, the most important method is `applyCodeTransformation`, which... applies the transformation! +Of course, it will also return the transformed code. +You can also see that the tool proposes some auxiliary output in the instantiation call of the `TransformationEditor` class (if you don't know this class, check the [third blog post](https://modularmoose.org/blog/2025-06-13-transformation-third/) on code transformation!). +Once a transformation is done, the transformation tool saves up information to highlight the transformations in the `TransformationEditor` using the `addedHighlights` and `removedHighlights` messages. +Another information the tool can give is available through the `transformationPointers` method, which is quite useful in the `TransformationEditor` when the entity being transformed is a class. +Using the button in the top right corner allows you to jump between entities who got a transformation, allowing you to easily verify those, even in the case of only a few transformations applied in a big class. + +## Try it yourself! + +With this, we've seen most of what transformation operations have to offer! + +Our little showcase tool allowed us to discover and manipulate those code transformation operations to create an actual transformation case. +Some types of operations are not covered here, however you can find the necessary explanations and examples through the tests and class documentations. +As you can see in this post, the way of using operations is also almost always the same independently of the type of the operation. + +Feel free to also try the code shown in this post in a Playground, or even to modify it to familiarize yourself with transformation operations! + +```smalltalk +tool := TransformationOperationShowcaseTool new. +tool applyAndViewTransformation +``` + +If you haven't used it earlier, quick reminder that the source code detailed in this blog post is available on this [repository](https://github.com/RomainDeg/Moose-BlogPost-Transformation). +If you followed the previous blog post series on code transformation, you may have recognized the repository! +Indeed, additionally to the code shown within this blog post, you will find on this branch the version of the tool shown in the previous blog post series, but replacing the previous transformation logic with transformation operations. + +## Conclusion + +In this blog post we used transformation operations, a new transformation system allowing for easier definition and seamless edits to keep comments and indentation after applying the transformation. + +The focus was on the operations system and not on everything else that matters when doing code transformation, like the querying and search through the Famix or FAST models to find the entities to transform, or how to apply the transformation on the actual source code of our software. +However, all of this was shown in the previous blog post series starting from [this post](https://modularmoose.org/blog/2024-04-01-transformation-first/). +The transformation operations system integrates perfectly with the other tools for code transformation, giving you everything necessary for your transformation needs! :smile: \ No newline at end of file diff --git a/src/content/docs/blog/img/posts/2025-10-15-transformation-operations/transformation_editor.png b/src/content/docs/blog/img/posts/2025-10-15-transformation-operations/transformation_editor.png new file mode 100644 index 0000000..52aa585 Binary files /dev/null and b/src/content/docs/blog/img/posts/2025-10-15-transformation-operations/transformation_editor.png differ diff --git a/src/content/docs/blog/img/posts/2025-10-15-transformation-operations/uml_blogpost.drawio.svg b/src/content/docs/blog/img/posts/2025-10-15-transformation-operations/uml_blogpost.drawio.svg new file mode 100644 index 0000000..ea95722 --- /dev/null +++ b/src/content/docs/blog/img/posts/2025-10-15-transformation-operations/uml_blogpost.drawio.svg @@ -0,0 +1,4 @@ + + + +
AbstractTransformationOperation
+ anchorNode : FASTJavaEntity
+ transformedNode : FASTJavaEntity
- buildNewNode : FASTJavaEntity
- getPosForSorting : int
- copyNode(FastJavaEntity) : FASTJavaEntity
+ anchorNode : FASTJavaEntity
+ anchorNode(FASTJavaEntity) : nil
+ transformedNode : FASTJavaEntity
+ transformedNode(FASTJavaEntity) : nil
DeleteOperation
 
+ nodeToDelete : FASTJavaEntity
+ nodeToDelete(FASTJavaEntity) : nil
ReplacingOperation
 
+ originalNode : FASTJavaEntity
+ originalNode(FASTJavaEntity) : nil
+ nodeToReplace : FASTJavaEntity
+ nodeToReplace(FASTJavaEntity) : nil
InsertionOperation
+ selector : ByteString
+ parentNode : FASTJavaEntity
+ parentNode(FASTJavaEntity) : nil
+ nodeToInsert : FASTJavaEntity
+ nodeToInsert(FASTJavaEntity) : nil
SignatureInsertionOperation
 
 
StatementInsertionOperation
+ insertBefore : Boolean
 
DeclarationInsertionOperation
 
 
\ No newline at end of file