From 093a48a718edf776176ec64e4bd104ca682b1735 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Wed, 28 May 2025 21:37:56 +1000 Subject: [PATCH 01/25] Add proposal --- text/3778-proc-macro-in-same-crate-as-app.md | 91 ++++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 text/3778-proc-macro-in-same-crate-as-app.md diff --git a/text/3778-proc-macro-in-same-crate-as-app.md b/text/3778-proc-macro-in-same-crate-as-app.md new file mode 100644 index 00000000000..b1fa7881519 --- /dev/null +++ b/text/3778-proc-macro-in-same-crate-as-app.md @@ -0,0 +1,91 @@ +- Feature Name: `proc-macro-in-same-crate-as-app` +- Start Date: 2025-5-29 + +tbd: +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +Have a new folder in a cargo project, called `proc-macro`. This would be like the `tests` directory in that it is alongside the source code. This would eliminate the need to create an extra crate for proc macros. + +# Motivation +[motivation]: #motivation + +A common thing to ask about proc macros when one is first learning them is: "Why on earth does it have to be in a separate crate?!" Of course, we eventually get to know that the reason is that proc macros are basically *compiler plugins*, meaning that they have to be compiled first, before the main code is compiled. So in summary, one needs to be compiled before the other. + +It doesn't have to be this way though, because we already have this mechanism of compiling orders – the example that come to mind is the `tests` directory. It relies on the `src` directory being built first, and likewise we could introduce a `proc-macro` directory that would be compiled before `src`. + +The motivation of having this new directory comes down to just convenience. This may sound crude at first, but convenience is a key part of any feature in software. It is known in UX design that every feature has an *interaction cost*: how much effort do I need to put in to use the feature? For example, a feature with low interaction cost, with text editor support, is renaming a variable. Just press F2 and type in a new name. What this provides is incredibly useful – without it, having a subpar variable/function name needed a high interaction cost, especially if it is used across multiple files, and as a result, we are discouraged to change variable names to make it better, when we have new retrospect. With a lower interaction cost, the renaming operation is greatly promoted, and leads to better code. + +This proposal aims smooth out the user experience when it comes to creating new proc macro, and achieve a similar effect to the F2 operation. It is important to emphasise that proc macros can dramatically simplify code, especially derive macros, but they a lot of the times aren't used because of all the extra hoops one has to get through. This would make proc macros (more of) "yet another feature", rather than a daunting one. + +An objection to this one might raise is "How much harder is typing in `cargo new` than `mkdir proc-macro`?" But we should consider if we would still use as much integration tests if the `tests` directory if it is required to be in a seperate crate. The answer is most likely less. This is because (1) having a new crate requires ceremony, like putting in a new dependency in cargo.toml, and (2) requires adding to the project structure. A *tiny* bit in lowering the interaction cost, even from 2 steps to 1, can greatly improve the user experience. + +In summary (TL;DR), the effort one needs to put in to use a feature is extremely important. Proc macros currently has a higher ceiling, needing one to create a whole new crate in order to use it, and lowering the ceiling, even just a little bit, could massively improve user experience. This proposal can lower it. + +# Explanation +[explanation]: #explanation + +Currently, we create a new proc macro as so: +1. Create a new crate +2. In its cargo.toml, specify that it is a proc macro crate +3. In the main project, add the crate as a dependency +4. Implement the proc macro in the new crate + +After this change, we create a new proc macro like this: +1. Create a new directory called `proc-macro` alongside your `src` directory +2. Implement the proc macro in a new file in `proc-macro`. + +To use the proc macro, simply import it via `crate::proc_macro`. +```rust +use crate::proc_macro::my_file::my_macro; +``` +Or, if the file happens to be `mod.rs`, you can access it directly after the `proc_macro` bit. + +## Proc Macro Libraries +Crates like `syn`, `quote`, and `proc-macro2`, would be included under `[dev-dependecies]` in the cargo.toml. (Perhaps we should put it in build dependencies? or a new dependency section for proc macros.) + +## How it would work in the implementation +Cargo would have to compile the `proc-macro` directory first, as a proc macro type (of course). Then, in compiling the main code, `crate::proc_macro::file_name::my_macro` would resolve the module to the file `/proc-macro/file_name.rs`. Alternatively, if the user uses `mod.rs`, it would be resolved from `crate::proc_macro::my_macro`. This would finally be passed into rustc. + +# Drawbacks +[drawbacks]: #drawbacks + +1. The proc macro directory cannot use functions from src. (but that was not possible before anyways) + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +> Have proc macro files marked `#![proc_macro_file]` to signal to cargo to compile it first. + +Since it would compile first, proc macro files cannot import functions in the main code. The problem is having it side-by-side to the rest of your code makes it seem like you could just import it, when you cannot. Having it as a seperate directory makes clear of this. + +> Eliminate the need for new proc macro files/folders entirely, have the compiler work out where the proc macros are and separate them. + +This would suffer from the same issue as the last alternative, plus being harder to implement. + +> Introspection + +Harder to implement, with less payoff. + +# Prior art +[prior-art]: #prior-art + +1. Zig comptime: metaprogramming code can sit directly next to application code. +2. Declarative macros: can sit side by side as well, but is less powerful. +3. Lisp macros: same as last two, except more powerful. +4. `tests` directory, and `build.rs`: compiled at a different time as the main code. +5. `Makefiles`, or other build systems: they allow for more customisability for when code is built. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +1. Should proc macro dependencies be listed under `[dev-dependencies]`, `[build-dependencies]`, or a new `[proc-macro-dependencies]` section? +2. Should we import like `crate::proc_macro::file::macro`, or via a new keyword, like `crate_macros::file::macro`? The latter would avoid name collisions, but might be more confusing. + +# Future possibilities +[future-possibilities]: #future-possibilities + +As described in the [motivation] section, this proposal is aimed to make the process of creating proc macros easier. So a natural extension of this is to remove the need of third-party crates like syn and proc-macro2. There is already an effort to implement quote, so they might be a possibility. \ No newline at end of file From 6400116415d1aafc446f741bd971346c447d1605 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Thu, 29 May 2025 16:59:06 +1000 Subject: [PATCH 02/25] Change "crate" to "package" --- ...3778-proc-macro-in-same-package-as-app.md} | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) rename text/{3778-proc-macro-in-same-crate-as-app.md => 3778-proc-macro-in-same-package-as-app.md} (76%) diff --git a/text/3778-proc-macro-in-same-crate-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md similarity index 76% rename from text/3778-proc-macro-in-same-crate-as-app.md rename to text/3778-proc-macro-in-same-package-as-app.md index b1fa7881519..7c93e1c854b 100644 --- a/text/3778-proc-macro-in-same-crate-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -1,5 +1,5 @@ -- Feature Name: `proc-macro-in-same-crate-as-app` -- Start Date: 2025-5-29 +- Feature Name: `proc-macro-in-same-package-as-app` +- Start Date: tbd tbd: - RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) @@ -8,12 +8,12 @@ tbd: # Summary [summary]: #summary -Have a new folder in a cargo project, called `proc-macro`. This would be like the `tests` directory in that it is alongside the source code. This would eliminate the need to create an extra crate for proc macros. +Have a new folder in a cargo project, called `proc-macro`. This would be like the `tests` directory in that it is alongside the source code. This would eliminate the need to create an extra package for proc macros. # Motivation [motivation]: #motivation -A common thing to ask about proc macros when one is first learning them is: "Why on earth does it have to be in a separate crate?!" Of course, we eventually get to know that the reason is that proc macros are basically *compiler plugins*, meaning that they have to be compiled first, before the main code is compiled. So in summary, one needs to be compiled before the other. +A common thing to ask about proc macros when one is first learning them is: "Why on earth does it have to be in a separate package?!" Of course, we eventually get to know that the reason is that proc macros are basically *compiler plugins*, meaning that they have to be compiled first, before the main code is compiled. So in summary, one needs to be compiled before the other. It doesn't have to be this way though, because we already have this mechanism of compiling orders – the example that come to mind is the `tests` directory. It relies on the `src` directory being built first, and likewise we could introduce a `proc-macro` directory that would be compiled before `src`. @@ -21,18 +21,18 @@ The motivation of having this new directory comes down to just convenience. This This proposal aims smooth out the user experience when it comes to creating new proc macro, and achieve a similar effect to the F2 operation. It is important to emphasise that proc macros can dramatically simplify code, especially derive macros, but they a lot of the times aren't used because of all the extra hoops one has to get through. This would make proc macros (more of) "yet another feature", rather than a daunting one. -An objection to this one might raise is "How much harder is typing in `cargo new` than `mkdir proc-macro`?" But we should consider if we would still use as much integration tests if the `tests` directory if it is required to be in a seperate crate. The answer is most likely less. This is because (1) having a new crate requires ceremony, like putting in a new dependency in cargo.toml, and (2) requires adding to the project structure. A *tiny* bit in lowering the interaction cost, even from 2 steps to 1, can greatly improve the user experience. +An objection to this one might raise is "How much harder is typing in `cargo new` than `mkdir proc-macro`?" But we should consider if we would still use as much integration tests if the `tests` directory if it is required to be in a seperate package. The answer is most likely less. This is because (1) having a new package requires ceremony, like putting in a new dependency in cargo.toml, and (2) requires adding to the project structure. A *tiny* bit in lowering the interaction cost, even from 2 steps to 1, can greatly improve the user experience. -In summary (TL;DR), the effort one needs to put in to use a feature is extremely important. Proc macros currently has a higher ceiling, needing one to create a whole new crate in order to use it, and lowering the ceiling, even just a little bit, could massively improve user experience. This proposal can lower it. +In summary (TL;DR), the effort one needs to put in to use a feature is extremely important. Proc macros currently has a higher ceiling, needing one to create a whole new package in order to use it, and lowering the ceiling, even just a little bit, could massively improve user experience. This proposal can lower it. # Explanation [explanation]: #explanation Currently, we create a new proc macro as so: -1. Create a new crate -2. In its cargo.toml, specify that it is a proc macro crate -3. In the main project, add the crate as a dependency -4. Implement the proc macro in the new crate +1. Create a new package +2. In its cargo.toml, specify that it is a proc macro package +3. In the main project, add the package as a dependency +4. Implement the proc macro in the new package After this change, we create a new proc macro like this: 1. Create a new directory called `proc-macro` alongside your `src` directory @@ -45,7 +45,7 @@ use crate::proc_macro::my_file::my_macro; Or, if the file happens to be `mod.rs`, you can access it directly after the `proc_macro` bit. ## Proc Macro Libraries -Crates like `syn`, `quote`, and `proc-macro2`, would be included under `[dev-dependecies]` in the cargo.toml. (Perhaps we should put it in build dependencies? or a new dependency section for proc macros.) +Libraries like `syn`, `quote`, and `proc-macro2`, would be included under `[dev-dependecies]` in the cargo.toml. (Perhaps we should put it in build dependencies? or a new dependency section for proc macros.) ## How it would work in the implementation Cargo would have to compile the `proc-macro` directory first, as a proc macro type (of course). Then, in compiling the main code, `crate::proc_macro::file_name::my_macro` would resolve the module to the file `/proc-macro/file_name.rs`. Alternatively, if the user uses `mod.rs`, it would be resolved from `crate::proc_macro::my_macro`. This would finally be passed into rustc. @@ -88,4 +88,4 @@ Harder to implement, with less payoff. # Future possibilities [future-possibilities]: #future-possibilities -As described in the [motivation] section, this proposal is aimed to make the process of creating proc macros easier. So a natural extension of this is to remove the need of third-party crates like syn and proc-macro2. There is already an effort to implement quote, so they might be a possibility. \ No newline at end of file +As described in the [motivation] section, this proposal is aimed to make the process of creating proc macros easier. So a natural extension of this is to remove the need of third-party libraries like syn and proc-macro2. There is already an effort to implement quote, so they might be a possibility. \ No newline at end of file From bd3762d1fb33477d7b936c18f8c0163647b9a7e4 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Fri, 30 May 2025 17:09:53 +1000 Subject: [PATCH 03/25] Implement feedback from pre-RFC --- text/3778-proc-macro-in-same-package-as-app.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 7c93e1c854b..626e9b266a6 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -17,6 +17,8 @@ A common thing to ask about proc macros when one is first learning them is: "Why It doesn't have to be this way though, because we already have this mechanism of compiling orders – the example that come to mind is the `tests` directory. It relies on the `src` directory being built first, and likewise we could introduce a `proc-macro` directory that would be compiled before `src`. +**To be absolutely clear**, this is not a proposal for same-*crate* proc macros (unlike previous proposals), but same-*package* proc macros. + The motivation of having this new directory comes down to just convenience. This may sound crude at first, but convenience is a key part of any feature in software. It is known in UX design that every feature has an *interaction cost*: how much effort do I need to put in to use the feature? For example, a feature with low interaction cost, with text editor support, is renaming a variable. Just press F2 and type in a new name. What this provides is incredibly useful – without it, having a subpar variable/function name needed a high interaction cost, especially if it is used across multiple files, and as a result, we are discouraged to change variable names to make it better, when we have new retrospect. With a lower interaction cost, the renaming operation is greatly promoted, and leads to better code. This proposal aims smooth out the user experience when it comes to creating new proc macro, and achieve a similar effect to the F2 operation. It is important to emphasise that proc macros can dramatically simplify code, especially derive macros, but they a lot of the times aren't used because of all the extra hoops one has to get through. This would make proc macros (more of) "yet another feature", rather than a daunting one. @@ -88,4 +90,8 @@ Harder to implement, with less payoff. # Future possibilities [future-possibilities]: #future-possibilities -As described in the [motivation] section, this proposal is aimed to make the process of creating proc macros easier. So a natural extension of this is to remove the need of third-party libraries like syn and proc-macro2. There is already an effort to implement quote, so they might be a possibility. \ No newline at end of file +As described in the [motivation] section, this proposal is aimed to make the process of creating proc macros easier. So a natural extension of this is to remove the need of third-party libraries like syn and proc-macro2. There is already an effort to implement quote, so they might be a possibility. + +Second, this might enable for some sort of `$crate` metavariable. + +Another possibility is possibly allowing for proc macro crates to export data structures, which would make writing certain things easier. \ No newline at end of file From 2abc88c8a00edd13e211ca9afb07b7df2f82cd8e Mon Sep 17 00:00:00 2001 From: ora-0 Date: Fri, 30 May 2025 17:24:55 +1000 Subject: [PATCH 04/25] General edits --- text/3778-proc-macro-in-same-package-as-app.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 626e9b266a6..0481ad8e069 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -8,18 +8,18 @@ tbd: # Summary [summary]: #summary -Have a new folder in a cargo project, called `proc-macro`. This would be like the `tests` directory in that it is alongside the source code. This would eliminate the need to create an extra package for proc macros. +Have a new folder in a cargo project, called `proc-macro`. This would be like the `tests` directory in that it is alongside the source code. It would eliminate the need to create an extra package for proc macros. # Motivation [motivation]: #motivation -A common thing to ask about proc macros when one is first learning them is: "Why on earth does it have to be in a separate package?!" Of course, we eventually get to know that the reason is that proc macros are basically *compiler plugins*, meaning that they have to be compiled first, before the main code is compiled. So in summary, one needs to be compiled before the other. +A common thing to ask about proc macros when first learning them is: "Why on earth does it have to be in a separate package?!" Of course, we eventually get to know that the reason is that proc macros are basically *compiler plugins*, meaning that they have to be compiled first, before the main code is compiled. So in summary, one needs to be compiled before the other. -It doesn't have to be this way though, because we already have this mechanism of compiling orders – the example that come to mind is the `tests` directory. It relies on the `src` directory being built first, and likewise we could introduce a `proc-macro` directory that would be compiled before `src`. +It doesn't have to be this way though, because we already have this mechanism of compiling one thing before another – for example, the `tests` directory. It relies on the `src` directory being built first, and likewise we could introduce a `proc-macro` directory that would compile before `src`. -**To be absolutely clear**, this is not a proposal for same-*crate* proc macros (unlike previous proposals), but same-*package* proc macros. +**To be absolutely clear**, this is not a proposal for same-*crate* proc macros (unlike previous proposals), but same-*package* proc macros: a much simpler problem. -The motivation of having this new directory comes down to just convenience. This may sound crude at first, but convenience is a key part of any feature in software. It is known in UX design that every feature has an *interaction cost*: how much effort do I need to put in to use the feature? For example, a feature with low interaction cost, with text editor support, is renaming a variable. Just press F2 and type in a new name. What this provides is incredibly useful – without it, having a subpar variable/function name needed a high interaction cost, especially if it is used across multiple files, and as a result, we are discouraged to change variable names to make it better, when we have new retrospect. With a lower interaction cost, the renaming operation is greatly promoted, and leads to better code. +The motivation of this new directory comes down to just convenience. This may sound crude at first, but convenience is a key part of any feature in software. It is known in UX design that every feature has an *interaction cost*: how much effort do I need to put in to use the feature? For example, a feature with low interaction cost, with text editor support, is renaming a variable. Just press F2 and type in a new name. What this provides is incredibly useful – without it, having a subpar variable/function name needed a high interaction cost, especially if it is used across multiple files, and as a result, we are discouraged to change variable names to make it better, when we have retrospect. With a lower interaction cost, the renaming operation is greatly promoted, and leads to better code. This proposal aims smooth out the user experience when it comes to creating new proc macro, and achieve a similar effect to the F2 operation. It is important to emphasise that proc macros can dramatically simplify code, especially derive macros, but they a lot of the times aren't used because of all the extra hoops one has to get through. This would make proc macros (more of) "yet another feature", rather than a daunting one. @@ -50,7 +50,7 @@ Or, if the file happens to be `mod.rs`, you can access it directly after the `pr Libraries like `syn`, `quote`, and `proc-macro2`, would be included under `[dev-dependecies]` in the cargo.toml. (Perhaps we should put it in build dependencies? or a new dependency section for proc macros.) ## How it would work in the implementation -Cargo would have to compile the `proc-macro` directory first, as a proc macro type (of course). Then, in compiling the main code, `crate::proc_macro::file_name::my_macro` would resolve the module to the file `/proc-macro/file_name.rs`. Alternatively, if the user uses `mod.rs`, it would be resolved from `crate::proc_macro::my_macro`. This would finally be passed into rustc. +Cargo would have to compile the `proc-macro` directory first, as a proc macro type (of course). Then, in compiling the main code, `crate::proc_macro::file_name::my_macro` would resolve the module to the file `proc-macro/file_name.rs`. Alternatively, if the user uses `mod.rs`, it would be resolved from `crate::proc_macro::my_macro`. This would finally be passed into rustc. # Drawbacks [drawbacks]: #drawbacks @@ -70,7 +70,7 @@ This would suffer from the same issue as the last alternative, plus being harder > Introspection -Harder to implement, with less payoff. +Harder to implement, with less payoff relative to the amount of work required. # Prior art [prior-art]: #prior-art @@ -79,7 +79,7 @@ Harder to implement, with less payoff. 2. Declarative macros: can sit side by side as well, but is less powerful. 3. Lisp macros: same as last two, except more powerful. 4. `tests` directory, and `build.rs`: compiled at a different time as the main code. -5. `Makefiles`, or other build systems: they allow for more customisability for when code is built. +5. `Makefiles`, or other build systems: they allow for customisability for when code is built. # Unresolved questions [unresolved-questions]: #unresolved-questions From cf956c9374c98febdea4b49ee518945f79ec0d7b Mon Sep 17 00:00:00 2001 From: ora-0 Date: Fri, 30 May 2025 17:47:58 +1000 Subject: [PATCH 05/25] Add info about pr --- text/3778-proc-macro-in-same-package-as-app.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 0481ad8e069..b394ea1d537 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -1,9 +1,7 @@ - Feature Name: `proc-macro-in-same-package-as-app` -- Start Date: tbd - -tbd: -- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) -- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) +- Start Date: 2025-05-30 +- RFC PR: [rust-lang/rfcs#3826](https://github.com/rust-lang/rfcs/pull/3826) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) tbd # Summary [summary]: #summary From 3ce11c20c9820fa081771d28517ed1f4daa3148f Mon Sep 17 00:00:00 2001 From: ora-0 Date: Sat, 31 May 2025 16:42:30 +1000 Subject: [PATCH 06/25] Revision 1: Implement feedback; Rethink `proc-macro` directory --- .../3778-proc-macro-in-same-package-as-app.md | 89 ++++++++++++++----- 1 file changed, 67 insertions(+), 22 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index b394ea1d537..a8722230795 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -6,22 +6,24 @@ # Summary [summary]: #summary -Have a new folder in a cargo project, called `proc-macro`. This would be like the `tests` directory in that it is alongside the source code. It would eliminate the need to create an extra package for proc macros. +Have a new target in a cargo project, called `proc-macro`. Its default location is in `src/macros.rs`. This would be like the `lib.rs` in that it is alongside the source code. It would eliminate the need to create an extra package for proc macros. # Motivation [motivation]: #motivation A common thing to ask about proc macros when first learning them is: "Why on earth does it have to be in a separate package?!" Of course, we eventually get to know that the reason is that proc macros are basically *compiler plugins*, meaning that they have to be compiled first, before the main code is compiled. So in summary, one needs to be compiled before the other. -It doesn't have to be this way though, because we already have this mechanism of compiling one thing before another – for example, the `tests` directory. It relies on the `src` directory being built first, and likewise we could introduce a `proc-macro` directory that would compile before `src`. +It doesn't have to be this way though, because we already have this mechanism of compiling one thing before another – for example, the `tests` directory. It relies on the `src` directory being built first, and likewise we could introduce a `proc-macro` target that would compile before `src`. **To be absolutely clear**, this is not a proposal for same-*crate* proc macros (unlike previous proposals), but same-*package* proc macros: a much simpler problem. -The motivation of this new directory comes down to just convenience. This may sound crude at first, but convenience is a key part of any feature in software. It is known in UX design that every feature has an *interaction cost*: how much effort do I need to put in to use the feature? For example, a feature with low interaction cost, with text editor support, is renaming a variable. Just press F2 and type in a new name. What this provides is incredibly useful – without it, having a subpar variable/function name needed a high interaction cost, especially if it is used across multiple files, and as a result, we are discouraged to change variable names to make it better, when we have retrospect. With a lower interaction cost, the renaming operation is greatly promoted, and leads to better code. +The motivation of this new target comes down to just convenience. This may sound crude at first, but convenience is a key part of any feature in software. It is known in UX design that every feature has an *interaction cost*: how much effort do I need to put in to use the feature? For example, a feature with low interaction cost, with text editor support, is renaming a variable. Just press F2 and type in a new name. What this provides is incredibly useful – without it, having a subpar variable/function name needed a high interaction cost, especially if it is used across multiple files, and as a result, we are discouraged to change variable names to make it better, when we have retrospect. With a lower interaction cost, the renaming operation is greatly promoted, and leads to better code. This proposal aims smooth out the user experience when it comes to creating new proc macro, and achieve a similar effect to the F2 operation. It is important to emphasise that proc macros can dramatically simplify code, especially derive macros, but they a lot of the times aren't used because of all the extra hoops one has to get through. This would make proc macros (more of) "yet another feature", rather than a daunting one. -An objection to this one might raise is "How much harder is typing in `cargo new` than `mkdir proc-macro`?" But we should consider if we would still use as much integration tests if the `tests` directory if it is required to be in a seperate package. The answer is most likely less. This is because (1) having a new package requires ceremony, like putting in a new dependency in cargo.toml, and (2) requires adding to the project structure. A *tiny* bit in lowering the interaction cost, even from 2 steps to 1, can greatly improve the user experience. +An objection to this one might raise is "How much harder is typing in `cargo new` than `touch macros.rs`?" But we should consider if we would still use as much integration tests if the `tests` directory if it is required to be in a seperate package. The answer is most likely less. This is because (1) having a new package requires ceremony, like putting in a new dependency in cargo.toml, and (2) requires adding to the project structure. A *tiny* bit in lowering the interaction cost, even from 2 steps to 1, can greatly improve the user experience. + +Another benefit is that a library developer don't have to manage two packages if one requires proc macros, and make them be in sync with each other. In summary (TL;DR), the effort one needs to put in to use a feature is extremely important. Proc macros currently has a higher ceiling, needing one to create a whole new package in order to use it, and lowering the ceiling, even just a little bit, could massively improve user experience. This proposal can lower it. @@ -35,38 +37,81 @@ Currently, we create a new proc macro as so: 4. Implement the proc macro in the new package After this change, we create a new proc macro like this: -1. Create a new directory called `proc-macro` alongside your `src` directory -2. Implement the proc macro in a new file in `proc-macro`. +1. Implement the proc macro in a new `macros.rs` in `proc-macro`. -To use the proc macro, simply import it via `crate::proc_macro`. +To use the proc macro, simply import it via `macros::*`. ```rust -use crate::proc_macro::my_file::my_macro; +use macros::my_macro; ``` -Or, if the file happens to be `mod.rs`, you can access it directly after the `proc_macro` bit. -## Proc Macro Libraries -Libraries like `syn`, `quote`, and `proc-macro2`, would be included under `[dev-dependecies]` in the cargo.toml. (Perhaps we should put it in build dependencies? or a new dependency section for proc macros.) +## An example +Suppose you are developing a library that would have normal functions as well as proc macros. The file structure would look like this: +``` +My-Amazing-Library +|---src +| |---lib.rs +| |---macros.rs +| |---common.rs +|---cargo.toml +``` +`common.rs` is a normal file that declares common data structures and functions. `macros.rs` defines macros, which will be made available to `lib.rs`. `lib.rs` can use the macros defined, and/or reexport the macros. + +Using code in `common.rs` in `macros.rs` is like how you would normally: +```rust +mod common; +use common::*; +``` + +## Cargo.toml configs +Libraries like `syn`, `quote`, and `proc-macro2`, would be included under `[build-dependecies]` in the cargo.toml. (Perhaps we should put it in a new dependency section for proc macros?) + +Like `tests` or `lib`, this would have its own `[proc-macro]` section in cargo.toml. + +Here are all the options available under it, the values set are its default. +```toml +[proc-macro] +name = "macros" +path = "src/macros.rs" +test = true +doctest = true +bench = false +doc = true +proc-macro = true # (cannot be changed) +``` + +To disable automatic finding, use: +```toml +[package] +autoprocmacro = false +``` ## How it would work in the implementation -Cargo would have to compile the `proc-macro` directory first, as a proc macro type (of course). Then, in compiling the main code, `crate::proc_macro::file_name::my_macro` would resolve the module to the file `proc-macro/file_name.rs`. Alternatively, if the user uses `mod.rs`, it would be resolved from `crate::proc_macro::my_macro`. This would finally be passed into rustc. +Then pass it into rustc with `--extern=macros=target/_profile_/deps/lib_____`. The process would occur after the compilation of `build.rs` to make metadata and files generated in OUT_DIR available, but before `lib.rs` to make macros available. This means `macros.rs` cannot use code in `lib.rs`, so the code would have to be factored out into a seperate file. # Drawbacks [drawbacks]: #drawbacks -1. The proc macro directory cannot use functions from src. (but that was not possible before anyways) +None at the moment # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives -> Have proc macro files marked `#![proc_macro_file]` to signal to cargo to compile it first. +1. Use `crate::macros::*` as the import path + +This would require changes to rustc, when there is a simpler solution. + +2. Have it within a `proc_macros` directory + +This was the original idea; but upon further consideration it turns out to be worse than the current. The justification of it over a file was: +> *Since it would compile first, proc macro files cannot import functions in the main code. The problem is having it side-by-side to the rest of your code makes it seem like you could just import it, when you cannot. Having it as a seperate directory makes clear of this.* -Since it would compile first, proc macro files cannot import functions in the main code. The problem is having it side-by-side to the rest of your code makes it seem like you could just import it, when you cannot. Having it as a seperate directory makes clear of this. +While it was true that proc macro files cannot import functions in the main code, it can import other modules, making the statement's merits false. -> Eliminate the need for new proc macro files/folders entirely, have the compiler work out where the proc macros are and separate them. +3. Eliminate the need for new proc macro files/folders entirely, have the compiler work out where the proc macros are and separate them. This would suffer from the same issue as the last alternative, plus being harder to implement. -> Introspection +4. Introspection Harder to implement, with less payoff relative to the amount of work required. @@ -82,14 +127,14 @@ Harder to implement, with less payoff relative to the amount of work required. # Unresolved questions [unresolved-questions]: #unresolved-questions -1. Should proc macro dependencies be listed under `[dev-dependencies]`, `[build-dependencies]`, or a new `[proc-macro-dependencies]` section? -2. Should we import like `crate::proc_macro::file::macro`, or via a new keyword, like `crate_macros::file::macro`? The latter would avoid name collisions, but might be more confusing. +1. Should proc macro dependencies be listed under `[build-dependencies]`, or a new `[proc-macro-dependencies]` section? +2. ~~Should we import like `crate::proc_macro::file::macro`, or via a new keyword, like `crate_macros::file::macro`? The latter would avoid name collisions, but might be more confusing.~~ # Future possibilities [future-possibilities]: #future-possibilities -As described in the [motivation] section, this proposal is aimed to make the process of creating proc macros easier. So a natural extension of this is to remove the need of third-party libraries like syn and proc-macro2. There is already an effort to implement quote, so they might be a possibility. +1. As described in the [motivation] section, this proposal is aimed to make the process of creating proc macros easier. So a natural extension of this is to remove the need of third-party libraries like syn and proc-macro2. There is already an effort to implement quote, so they might be a possibility. -Second, this might enable for some sort of `$crate` metavariable. +2. This might enable for some sort of `$crate` metavariable. -Another possibility is possibly allowing for proc macro crates to export data structures, which would make writing certain things easier. \ No newline at end of file +3. Enabling multiple lib targets. \ No newline at end of file From 269863b614c256122798f4f22550504ce00a44f4 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Sun, 1 Jun 2025 17:10:12 +1000 Subject: [PATCH 07/25] Revision 2: Rename macro.rs to procmacro.rs; Add detail in example and implementation details --- .../3778-proc-macro-in-same-package-as-app.md | 49 +++++++++++++++---- 1 file changed, 40 insertions(+), 9 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index a8722230795..3cad8bf2d4d 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -6,7 +6,7 @@ # Summary [summary]: #summary -Have a new target in a cargo project, called `proc-macro`. Its default location is in `src/macros.rs`. This would be like the `lib.rs` in that it is alongside the source code. It would eliminate the need to create an extra package for proc macros. +Have a new target in a cargo project, called `proc-macro`. Its default location is in `src/procmacros.rs`. This would be like the `lib.rs` in that it is alongside the source code. It would eliminate the need to create an extra package for proc macros. # Motivation [motivation]: #motivation @@ -21,7 +21,7 @@ The motivation of this new target comes down to just convenience. This may sound This proposal aims smooth out the user experience when it comes to creating new proc macro, and achieve a similar effect to the F2 operation. It is important to emphasise that proc macros can dramatically simplify code, especially derive macros, but they a lot of the times aren't used because of all the extra hoops one has to get through. This would make proc macros (more of) "yet another feature", rather than a daunting one. -An objection to this one might raise is "How much harder is typing in `cargo new` than `touch macros.rs`?" But we should consider if we would still use as much integration tests if the `tests` directory if it is required to be in a seperate package. The answer is most likely less. This is because (1) having a new package requires ceremony, like putting in a new dependency in cargo.toml, and (2) requires adding to the project structure. A *tiny* bit in lowering the interaction cost, even from 2 steps to 1, can greatly improve the user experience. +An objection to this one might raise is "How much harder is typing in `cargo new` than `touch procmacros.rs`?" But we should consider if we would still use as much integration tests if the `tests` directory if it is required to be in a seperate package. The answer is most likely less. This is because (1) having a new package requires ceremony, like putting in a new dependency in cargo.toml, and (2) requires adding to the project structure. A *tiny* bit in lowering the interaction cost, even from 2 steps to 1, can greatly improve the user experience. Another benefit is that a library developer don't have to manage two packages if one requires proc macros, and make them be in sync with each other. @@ -37,31 +37,50 @@ Currently, we create a new proc macro as so: 4. Implement the proc macro in the new package After this change, we create a new proc macro like this: -1. Implement the proc macro in a new `macros.rs` in `proc-macro`. +1. Implement the proc macro in a new `procmacros.rs` in `proc-macro`. +To build only the macro, use: +```console +$ cargo build --macros +``` + +## Importing +[importing]: #importing To use the proc macro, simply import it via `macros::*`. ```rust use macros::my_macro; ``` +Note that macros is only available to inside the package (i.e. bin, lib, examples...). This means that one would have to reexport the macros in `lib.rs` in order for users of a library to use it. It would still be available in `main.rs`, `tests`, `examples`, etc, though. + ## An example Suppose you are developing a library that would have normal functions as well as proc macros. The file structure would look like this: ``` My-Amazing-Library |---src | |---lib.rs -| |---macros.rs +| |---procmacros.rs | |---common.rs |---cargo.toml ``` -`common.rs` is a normal file that declares common data structures and functions. `macros.rs` defines macros, which will be made available to `lib.rs`. `lib.rs` can use the macros defined, and/or reexport the macros. +`common.rs` is a normal file that declares common data structures and functions. `procmacros.rs` defines macros, which will be made available to `lib.rs`. `lib.rs` can use the macros defined, and reexport the macros to make it available to anyone using the library. -Using code in `common.rs` in `macros.rs` is like how you would normally: +Using code in `common.rs` in `procmacros.rs` is like how you would normally: ```rust mod common; use common::*; ``` +Now, to make the macros available, reexport them, and if you want gate a macro behind a feature flag, it would be like how you would normally also, with cfg: +```rust +// in lib.rs +#[cfg(feature = "my_feature")] +pub use macros::a_niche_macro; +pub use macros::a_common_macro; // (not gated) +``` + +Finally, testing is also how you would expect it. It would be made available to the `tests` directory. ([importing]) + ## Cargo.toml configs Libraries like `syn`, `quote`, and `proc-macro2`, would be included under `[build-dependecies]` in the cargo.toml. (Perhaps we should put it in a new dependency section for proc macros?) @@ -71,7 +90,7 @@ Here are all the options available under it, the values set are its default. ```toml [proc-macro] name = "macros" -path = "src/macros.rs" +path = "src/procmacros.rs" test = true doctest = true bench = false @@ -85,8 +104,19 @@ To disable automatic finding, use: autoprocmacro = false ``` -## How it would work in the implementation -Then pass it into rustc with `--extern=macros=target/_profile_/deps/lib_____`. The process would occur after the compilation of `build.rs` to make metadata and files generated in OUT_DIR available, but before `lib.rs` to make macros available. This means `macros.rs` cannot use code in `lib.rs`, so the code would have to be factored out into a seperate file. +## Implemention Details +The package targets would be compiled in the following order: +1. `build` +2. `macros` +3. `lib` +4. `bin`s +5. ... + +During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(config!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. + +Any libraries to be linked, as specified in `build.rs` via stdout, would be linked and made available in the `macros`. + +The compiled macros crate would be passed into rustc with `--extern=macros=target/_profile_/deps/lib_____` when compiling the other crates. # Drawbacks [drawbacks]: #drawbacks @@ -128,6 +158,7 @@ Harder to implement, with less payoff relative to the amount of work required. [unresolved-questions]: #unresolved-questions 1. Should proc macro dependencies be listed under `[build-dependencies]`, or a new `[proc-macro-dependencies]` section? +2. What case should `procmacros.rs` be? No spaces, kebab case, or snake? Having no spaces would be the most agnostic solution 2. ~~Should we import like `crate::proc_macro::file::macro`, or via a new keyword, like `crate_macros::file::macro`? The latter would avoid name collisions, but might be more confusing.~~ # Future possibilities From 001a769b6f0747aa5712ecf5fd172074ad72d5b1 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Tue, 3 Jun 2025 18:48:11 +1000 Subject: [PATCH 08/25] fix capitalisation Co-authored-by: Ed Page --- text/3778-proc-macro-in-same-package-as-app.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 3cad8bf2d4d..172afe2f21c 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -61,7 +61,7 @@ My-Amazing-Library | |---lib.rs | |---procmacros.rs | |---common.rs -|---cargo.toml +|---Cargo.toml ``` `common.rs` is a normal file that declares common data structures and functions. `procmacros.rs` defines macros, which will be made available to `lib.rs`. `lib.rs` can use the macros defined, and reexport the macros to make it available to anyone using the library. From a4cc17738a10d1b5741d450f11deae89044ee08d Mon Sep 17 00:00:00 2001 From: ora-0 Date: Tue, 3 Jun 2025 18:48:33 +1000 Subject: [PATCH 09/25] change config to cfg Co-authored-by: Ed Page --- text/3778-proc-macro-in-same-package-as-app.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 172afe2f21c..a85dba65a5d 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -112,7 +112,7 @@ The package targets would be compiled in the following order: 4. `bin`s 5. ... -During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(config!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. +During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(cfg!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. Any libraries to be linked, as specified in `build.rs` via stdout, would be linked and made available in the `macros`. From dad0bef95c9a9fe5430b71dc5cc36854763c13e2 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Tue, 3 Jun 2025 16:45:41 +1000 Subject: [PATCH 10/25] Change default location to proc-macro/lib.rs --- .../3778-proc-macro-in-same-package-as-app.md | 34 +++++++------------ 1 file changed, 13 insertions(+), 21 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index a85dba65a5d..9c79aa3160b 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -6,7 +6,7 @@ # Summary [summary]: #summary -Have a new target in a cargo project, called `proc-macro`. Its default location is in `src/procmacros.rs`. This would be like the `lib.rs` in that it is alongside the source code. It would eliminate the need to create an extra package for proc macros. +Have a new target in a cargo project, called `proc-macro`. Its default location is in `proc-macro/lib.rs`. This would be like the `tests` directory in that it is alongside the source code. It would eliminate the need to create an extra package for proc macros. # Motivation [motivation]: #motivation @@ -21,7 +21,7 @@ The motivation of this new target comes down to just convenience. This may sound This proposal aims smooth out the user experience when it comes to creating new proc macro, and achieve a similar effect to the F2 operation. It is important to emphasise that proc macros can dramatically simplify code, especially derive macros, but they a lot of the times aren't used because of all the extra hoops one has to get through. This would make proc macros (more of) "yet another feature", rather than a daunting one. -An objection to this one might raise is "How much harder is typing in `cargo new` than `touch procmacros.rs`?" But we should consider if we would still use as much integration tests if the `tests` directory if it is required to be in a seperate package. The answer is most likely less. This is because (1) having a new package requires ceremony, like putting in a new dependency in cargo.toml, and (2) requires adding to the project structure. A *tiny* bit in lowering the interaction cost, even from 2 steps to 1, can greatly improve the user experience. +An objection to this one might raise is "How much harder is typing in `cargo new` than `mkdir proc-macro`?" But we should consider if we would still use as much integration tests if the `tests` directory if it is required to be in a seperate package. The answer is most likely less. This is because (1) having a new package requires ceremony, like putting in a new dependency in cargo.toml, and (2) requires adding to the project structure. A *tiny* bit in lowering the interaction cost, even from 2 steps to 1, can greatly improve the user experience. Another benefit is that a library developer don't have to manage two packages if one requires proc macros, and make them be in sync with each other. @@ -37,7 +37,8 @@ Currently, we create a new proc macro as so: 4. Implement the proc macro in the new package After this change, we create a new proc macro like this: -1. Implement the proc macro in a new `procmacros.rs` in `proc-macro`. +1. Create a new folder in the root of the project called `proc-macro` +2. Implement the proc macro in a new `lib.rs` in the new folder. To build only the macro, use: ```console @@ -57,23 +58,17 @@ Note that macros is only available to inside the package (i.e. bin, lib, example Suppose you are developing a library that would have normal functions as well as proc macros. The file structure would look like this: ``` My-Amazing-Library +|---proc-macro +| |---lib.rs |---src | |---lib.rs -| |---procmacros.rs -| |---common.rs -|---Cargo.toml -``` -`common.rs` is a normal file that declares common data structures and functions. `procmacros.rs` defines macros, which will be made available to `lib.rs`. `lib.rs` can use the macros defined, and reexport the macros to make it available to anyone using the library. - -Using code in `common.rs` in `procmacros.rs` is like how you would normally: -```rust -mod common; -use common::*; +|---cargo.toml ``` +`proc-macro/lib.rs` defines macros, which will be made available to `src/lib.rs`. `src/lib.rs` can use the macros defined, and reexport the macros to make it available to anyone using the library. Now, to make the macros available, reexport them, and if you want gate a macro behind a feature flag, it would be like how you would normally also, with cfg: ```rust -// in lib.rs +// in src/lib.rs #[cfg(feature = "my_feature")] pub use macros::a_niche_macro; pub use macros::a_common_macro; // (not gated) @@ -90,7 +85,7 @@ Here are all the options available under it, the values set are its default. ```toml [proc-macro] name = "macros" -path = "src/procmacros.rs" +path = "proc-macro/lib.rs" test = true doctest = true bench = false @@ -130,12 +125,9 @@ None at the moment This would require changes to rustc, when there is a simpler solution. -2. Have it within a `proc_macros` directory - -This was the original idea; but upon further consideration it turns out to be worse than the current. The justification of it over a file was: -> *Since it would compile first, proc macro files cannot import functions in the main code. The problem is having it side-by-side to the rest of your code makes it seem like you could just import it, when you cannot. Having it as a seperate directory makes clear of this.* +2. Have it within `src/proc-macro.rs` -While it was true that proc macro files cannot import functions in the main code, it can import other modules, making the statement's merits false. +The problem with this is the confusion it creates for users. Someone looking through `src/` aren't able to see which files are part of the library, or part of proc macro. In addition, having it within the same directory have led some users, like with `lib.rs` and `main.rs`, to use `mod` for importing when they meant `use` from the library. 3. Eliminate the need for new proc macro files/folders entirely, have the compiler work out where the proc macros are and separate them. @@ -158,7 +150,7 @@ Harder to implement, with less payoff relative to the amount of work required. [unresolved-questions]: #unresolved-questions 1. Should proc macro dependencies be listed under `[build-dependencies]`, or a new `[proc-macro-dependencies]` section? -2. What case should `procmacros.rs` be? No spaces, kebab case, or snake? Having no spaces would be the most agnostic solution +2. What case should `proc-macro/` be? No spaces, kebab case, or snake? Having no spaces would be the most agnostic solution 2. ~~Should we import like `crate::proc_macro::file::macro`, or via a new keyword, like `crate_macros::file::macro`? The latter would avoid name collisions, but might be more confusing.~~ # Future possibilities From f4ef425acd1ceac6d5d6e6ecac5e7a7f2707d2c7 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Tue, 3 Jun 2025 18:55:45 +1000 Subject: [PATCH 11/25] Split explanation into guide-level and reference-level --- .../3778-proc-macro-in-same-package-as-app.md | 36 ++++++++++--------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 9c79aa3160b..1dad8a2806a 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -27,8 +27,8 @@ Another benefit is that a library developer don't have to manage two packages if In summary (TL;DR), the effort one needs to put in to use a feature is extremely important. Proc macros currently has a higher ceiling, needing one to create a whole new package in order to use it, and lowering the ceiling, even just a little bit, could massively improve user experience. This proposal can lower it. -# Explanation -[explanation]: #explanation +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation Currently, we create a new proc macro as so: 1. Create a new package @@ -76,6 +76,24 @@ pub use macros::a_common_macro; // (not gated) Finally, testing is also how you would expect it. It would be made available to the `tests` directory. ([importing]) +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +The package targets would be compiled in the following order: +1. `build` +2. `macros` +3. `lib` +4. `bin`s +5. ... + +The macros would be available to all targets built afterwards. + +During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(config!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. + +Any libraries to be linked, as specified in `build.rs` via stdout, would be linked and made available in the `macros`. + +The compiled macros crate would be passed into rustc with `--extern=macros=target/_profile_/deps/lib_____` when compiling the other crates. + ## Cargo.toml configs Libraries like `syn`, `quote`, and `proc-macro2`, would be included under `[build-dependecies]` in the cargo.toml. (Perhaps we should put it in a new dependency section for proc macros?) @@ -99,20 +117,6 @@ To disable automatic finding, use: autoprocmacro = false ``` -## Implemention Details -The package targets would be compiled in the following order: -1. `build` -2. `macros` -3. `lib` -4. `bin`s -5. ... - -During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(cfg!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. - -Any libraries to be linked, as specified in `build.rs` via stdout, would be linked and made available in the `macros`. - -The compiled macros crate would be passed into rustc with `--extern=macros=target/_profile_/deps/lib_____` when compiling the other crates. - # Drawbacks [drawbacks]: #drawbacks From 6e1111ab68484d91e77209d6baaf0394b02c7e0c Mon Sep 17 00:00:00 2001 From: ora-0 Date: Tue, 3 Jun 2025 20:07:07 +1000 Subject: [PATCH 12/25] Specify visibility of `macros` within the reference-level explanation section --- text/3778-proc-macro-in-same-package-as-app.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 1dad8a2806a..a09ad3f3a92 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -86,7 +86,7 @@ The package targets would be compiled in the following order: 4. `bin`s 5. ... -The macros would be available to all targets built afterwards. +The macros would be available to all targets built afterwards. Exports of `macros` is only available inside the package, so any publicly available ones need to be reexported in `lib`. During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(config!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. From 3b34d6873d93692cdbfe10d86610198b38733753 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Tue, 3 Jun 2025 20:19:30 +1000 Subject: [PATCH 13/25] Reform the linking behaviour specified by `build.rs` with `macros` --- text/3778-proc-macro-in-same-package-as-app.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index a09ad3f3a92..875d9fe3f50 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -90,7 +90,7 @@ The macros would be available to all targets built afterwards. Exports of `macro During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(config!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. -Any libraries to be linked, as specified in `build.rs` via stdout, would be linked and made available in the `macros`. +Any libraries to be linked, as specified in `build.rs` via stdout, are not to be available to `macros`. In addition, linker arguments can be passed through `cargo::rustc-link-arg-macros=FLAG` via stdout of `build.rs`. The compiled macros crate would be passed into rustc with `--extern=macros=target/_profile_/deps/lib_____` when compiling the other crates. From 2a98d110c5bae28ead145aadcfa480afa564928d Mon Sep 17 00:00:00 2001 From: ora-0 Date: Tue, 3 Jun 2025 20:30:09 +1000 Subject: [PATCH 14/25] Discover drawbacks --- text/3778-proc-macro-in-same-package-as-app.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 875d9fe3f50..0262e5a5f68 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -120,7 +120,8 @@ autoprocmacro = false # Drawbacks [drawbacks]: #drawbacks -None at the moment +1. Added complexity - Somewhat increases maintainance cost of cargo +2. Migrations - Existing crates now need to migrate to the new system, taking time, and it may cause some exisiting code that's always using the latest version of libraries to break. # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives From 9fd805f03fe0dfff82a30a86d646b0af902c5a9b Mon Sep 17 00:00:00 2001 From: ora-0 Date: Wed, 4 Jun 2025 16:33:58 +1000 Subject: [PATCH 15/25] Reapply config -> cfg Co-authored-by: Ed Page --- text/3778-proc-macro-in-same-package-as-app.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 0262e5a5f68..c579c2ef5e3 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -88,7 +88,7 @@ The package targets would be compiled in the following order: The macros would be available to all targets built afterwards. Exports of `macros` is only available inside the package, so any publicly available ones need to be reexported in `lib`. -During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(config!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. +During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(cfg!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. Any libraries to be linked, as specified in `build.rs` via stdout, are not to be available to `macros`. In addition, linker arguments can be passed through `cargo::rustc-link-arg-macros=FLAG` via stdout of `build.rs`. From 4731ce19bc7cfd9aca9550c06254e061c3ab5725 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Wed, 4 Jun 2025 17:02:10 +1000 Subject: [PATCH 16/25] Add drawback relating to other build systems --- text/3778-proc-macro-in-same-package-as-app.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index c579c2ef5e3..418ff58ed38 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -122,6 +122,7 @@ autoprocmacro = false 1. Added complexity - Somewhat increases maintainance cost of cargo 2. Migrations - Existing crates now need to migrate to the new system, taking time, and it may cause some exisiting code that's always using the latest version of libraries to break. +3. Build systems that aren't Cargo needs to update to keep up with this feature # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives From 1488bdbc70f2ab4031dfb42e52f9943f4447f88d Mon Sep 17 00:00:00 2001 From: ora-0 Date: Wed, 4 Jun 2025 17:59:48 +1000 Subject: [PATCH 17/25] Add detail for environment variables --- .../3778-proc-macro-in-same-package-as-app.md | 29 +++++++++++++++++-- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 418ff58ed38..f7afa668a91 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -88,12 +88,35 @@ The package targets would be compiled in the following order: The macros would be available to all targets built afterwards. Exports of `macros` is only available inside the package, so any publicly available ones need to be reexported in `lib`. -During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(cfg!(proc_macro))` would be ok in the macros crate), as well as the `CARGO_CFG_PROC_MACRO` env variable. The `OUT_DIR` environment variable would be available, with all other usually available variables. - -Any libraries to be linked, as specified in `build.rs` via stdout, are not to be available to `macros`. In addition, linker arguments can be passed through `cargo::rustc-link-arg-macros=FLAG` via stdout of `build.rs`. +Any libraries to be linked, as specified in `build.rs` via stdout, are not to be available to `macros`. In addition, linker arguments can be passed through `cargo::rustc-link-arg-macros=FLAG` via stdout of `build.rs`. The compiled macros crate would be passed into rustc with `--extern=macros=target/_profile_/deps/lib_____` when compiling the other crates. +## Cfg and Environment Variables +During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(cfg!(proc_macro))` would be ok in the macros crate). + +As well as those it, the following environment variables are set. For conciseness, this RFC will not attempt to outline the use of all environment variables. Refer to the [documentation](https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates). +- `CARGO_CFG_PROC_MACRO` to 1. +- `CARGO` +- `CARGO_MANIFEST_DIR` +- `CARGO_MANIFEST_PATH` +- `CARGO_PKG_VERSION` +- `CARGO_PKG_VERSION_MAJOR` +- `CARGO_PKG_VERSION_MINOR` +- `CARGO_PKG_VERSION_PATCH` +- `CARGO_PKG_VERSION_PRE` +- `CARGO_PKG_AUTHORS` +- `CARGO_PKG_NAME` +- `CARGO_PKG_DESCRIPTION` +- `CARGO_PKG_HOMEPAGE` +- `CARGO_PKG_REPOSITORY` +- `CARGO_PKG_LICENSE` +- `CARGO_PKG_LICENSE_FILE` +- `CARGO_PKG_RUST_VERSION` +- `CARGO_PKG_README` +- `OUT_DIR` +- `CARGO_PRIMARY_PACKAGE` + ## Cargo.toml configs Libraries like `syn`, `quote`, and `proc-macro2`, would be included under `[build-dependecies]` in the cargo.toml. (Perhaps we should put it in a new dependency section for proc macros?) From 904a6873f24442c7c59c1300a396066133959e1f Mon Sep 17 00:00:00 2001 From: ora-0 Date: Wed, 4 Jun 2025 21:05:24 +1000 Subject: [PATCH 18/25] Include changes to the cli --- text/3778-proc-macro-in-same-package-as-app.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index f7afa668a91..11f0037d086 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -140,6 +140,11 @@ To disable automatic finding, use: autoprocmacro = false ``` +## Cargo CLI Additions +- `cargo build --macros` – Compile `macros` only +- `cargo build --all-targets` – Equivalent to specifying `--lib --bins --tests --benches --examples --macros` +- `cargo test --macros` would not be not possible + # Drawbacks [drawbacks]: #drawbacks From 2f47429fa5278e81abb7cacc8829350b3feb41d9 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Wed, 4 Jun 2025 21:19:05 +1000 Subject: [PATCH 19/25] Specify building behaviour in target/ --- text/3778-proc-macro-in-same-package-as-app.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 11f0037d086..20df5ced17e 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -90,7 +90,7 @@ The macros would be available to all targets built afterwards. Exports of `macro Any libraries to be linked, as specified in `build.rs` via stdout, are not to be available to `macros`. In addition, linker arguments can be passed through `cargo::rustc-link-arg-macros=FLAG` via stdout of `build.rs`. -The compiled macros crate would be passed into rustc with `--extern=macros=target/_profile_/deps/lib_____` when compiling the other crates. +Any artifacts created during compilation are to be in `target/_profile_/deps`, as usual. The compiled macros crate would be passed into rustc with `--extern=macros=target/_profile_/deps/lib_____` when compiling the other crates. Finally, the compiled lib_____ would be put in `target/macros/`, along with its `.d` file. ## Cfg and Environment Variables During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(cfg!(proc_macro))` would be ok in the macros crate). From 1dc76b1e18ab06b1b17f7bd44be91a1aecd29146 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Wed, 4 Jun 2025 21:19:48 +1000 Subject: [PATCH 20/25] Remove last point in #future-possibilities --- text/3778-proc-macro-in-same-package-as-app.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 20df5ced17e..391ec879d9d 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -192,6 +192,4 @@ Harder to implement, with less payoff relative to the amount of work required. 1. As described in the [motivation] section, this proposal is aimed to make the process of creating proc macros easier. So a natural extension of this is to remove the need of third-party libraries like syn and proc-macro2. There is already an effort to implement quote, so they might be a possibility. -2. This might enable for some sort of `$crate` metavariable. - -3. Enabling multiple lib targets. \ No newline at end of file +2. This might enable for some sort of `$crate` metavariable. \ No newline at end of file From cfb4c32a0431d48ed9a1ede5fa367d1f94fdcac4 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Wed, 4 Jun 2025 21:25:39 +1000 Subject: [PATCH 21/25] Move current behaviour to #motivations --- text/3778-proc-macro-in-same-package-as-app.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 391ec879d9d..1d8e6e98796 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -13,6 +13,14 @@ Have a new target in a cargo project, called `proc-macro`. Its default location A common thing to ask about proc macros when first learning them is: "Why on earth does it have to be in a separate package?!" Of course, we eventually get to know that the reason is that proc macros are basically *compiler plugins*, meaning that they have to be compiled first, before the main code is compiled. So in summary, one needs to be compiled before the other. +Currently, we create a new proc macro as so: +1. Create a new package +2. In its cargo.toml, specify that it is a proc macro package +3. In the main project, add the package as a dependency +4. Implement the proc macro in the new package + +While this doesn't seem like a lot, with reasons explained later, could actually be making code worse. + It doesn't have to be this way though, because we already have this mechanism of compiling one thing before another – for example, the `tests` directory. It relies on the `src` directory being built first, and likewise we could introduce a `proc-macro` target that would compile before `src`. **To be absolutely clear**, this is not a proposal for same-*crate* proc macros (unlike previous proposals), but same-*package* proc macros: a much simpler problem. @@ -30,12 +38,6 @@ In summary (TL;DR), the effort one needs to put in to use a feature is extremely # Guide-level explanation [guide-level-explanation]: #guide-level-explanation -Currently, we create a new proc macro as so: -1. Create a new package -2. In its cargo.toml, specify that it is a proc macro package -3. In the main project, add the package as a dependency -4. Implement the proc macro in the new package - After this change, we create a new proc macro like this: 1. Create a new folder in the root of the project called `proc-macro` 2. Implement the proc macro in a new `lib.rs` in the new folder. From c7f34c58a8a4d7613359b6434c4d1ab45c155c43 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Thu, 5 Jun 2025 16:14:25 +1000 Subject: [PATCH 22/25] Allow `cargo test --macros` --- text/3778-proc-macro-in-same-package-as-app.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 1d8e6e98796..a1af94cd5e7 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -145,7 +145,11 @@ autoprocmacro = false ## Cargo CLI Additions - `cargo build --macros` – Compile `macros` only - `cargo build --all-targets` – Equivalent to specifying `--lib --bins --tests --benches --examples --macros` -- `cargo test --macros` would not be not possible +- `cargo test --macros` – Test `macros` only + +## Documentation + +There would be a new item listed under "Crates" of the sidebar, for the new crate. This shoul # Drawbacks [drawbacks]: #drawbacks From 3f94b5b8214c49df37653eee2007b1d822a4fec0 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Thu, 5 Jun 2025 16:14:59 +1000 Subject: [PATCH 23/25] Remove `CARGO_CFG_PROC_MACRO` --- text/3778-proc-macro-in-same-package-as-app.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index a1af94cd5e7..7ad51e4be58 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -98,7 +98,6 @@ Any artifacts created during compilation are to be in `target/_profile_/deps`, a During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(cfg!(proc_macro))` would be ok in the macros crate). As well as those it, the following environment variables are set. For conciseness, this RFC will not attempt to outline the use of all environment variables. Refer to the [documentation](https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates). -- `CARGO_CFG_PROC_MACRO` to 1. - `CARGO` - `CARGO_MANIFEST_DIR` - `CARGO_MANIFEST_PATH` From ff8904062ef97ff6884410d3e174049b885070a7 Mon Sep 17 00:00:00 2001 From: ora-0 Date: Thu, 5 Jun 2025 16:43:42 +1000 Subject: [PATCH 24/25] Prefer `proc-macro` over `macros` when possible --- .../3778-proc-macro-in-same-package-as-app.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 7ad51e4be58..58587504ad0 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -44,7 +44,7 @@ After this change, we create a new proc macro like this: To build only the macro, use: ```console -$ cargo build --macros +$ cargo build --proc-macro ``` ## Importing @@ -72,8 +72,8 @@ Now, to make the macros available, reexport them, and if you want gate a macro b ```rust // in src/lib.rs #[cfg(feature = "my_feature")] -pub use macros::a_niche_macro; -pub use macros::a_common_macro; // (not gated) +pub use proc_macro::a_niche_macro; +pub use proc_macro::a_common_macro; // (not gated) ``` Finally, testing is also how you would expect it. It would be made available to the `tests` directory. ([importing]) @@ -83,19 +83,19 @@ Finally, testing is also how you would expect it. It would be made available to The package targets would be compiled in the following order: 1. `build` -2. `macros` +2. `proc-macro` 3. `lib` 4. `bin`s 5. ... -The macros would be available to all targets built afterwards. Exports of `macros` is only available inside the package, so any publicly available ones need to be reexported in `lib`. +The macros would be available to all targets built afterwards. Exports of `proc-macro` is only available inside the package, so any publicly available ones need to be reexported in `lib`. -Any libraries to be linked, as specified in `build.rs` via stdout, are not to be available to `macros`. In addition, linker arguments can be passed through `cargo::rustc-link-arg-macros=FLAG` via stdout of `build.rs`. +Any libraries to be linked, as specified in `build.rs` via stdout, are not to be available to `proc-macro`. In addition, linker arguments can be passed through `cargo::rustc-link-arg-proc-macro=FLAG` via stdout of `build.rs`. -Any artifacts created during compilation are to be in `target/_profile_/deps`, as usual. The compiled macros crate would be passed into rustc with `--extern=macros=target/_profile_/deps/lib_____` when compiling the other crates. Finally, the compiled lib_____ would be put in `target/macros/`, along with its `.d` file. +Any artifacts created during compilation are to be in `target/_profile_/deps`, as usual. The compiled macros crate would be passed into rustc with `--extern=proc_macro=target/_profile_/deps/lib_____` when compiling the other crates. Finally, the compiled lib_____ would be put in `target/proc-macro/`, along with its `.d` file. ## Cfg and Environment Variables -During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(cfg!(proc_macro))` would be ok in the macros crate). +During compilation, it would set the `proc_macro` cfg variable (i.e. `assert!(cfg!(proc_macro))` would be ok in the `proc-macro` crate). As well as those it, the following environment variables are set. For conciseness, this RFC will not attempt to outline the use of all environment variables. Refer to the [documentation](https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates). - `CARGO` @@ -142,9 +142,9 @@ autoprocmacro = false ``` ## Cargo CLI Additions -- `cargo build --macros` – Compile `macros` only -- `cargo build --all-targets` – Equivalent to specifying `--lib --bins --tests --benches --examples --macros` -- `cargo test --macros` – Test `macros` only +- `cargo build --proc-macro` – Compile `proc-macro` only +- `cargo build --all-targets` – Equivalent to specifying `--lib --bins --tests --benches --examples --proc-macro` +- `cargo test --proc-macro` – Test `proc-macro` only ## Documentation @@ -160,7 +160,7 @@ There would be a new item listed under "Crates" of the sidebar, for the new crat # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives -1. Use `crate::macros::*` as the import path +1. Use `crate::proc_macro::*` as the import path This would require changes to rustc, when there is a simpler solution. From ad1964fc01f3042807ab8e91d82263125ad5f2da Mon Sep 17 00:00:00 2001 From: ora-0 Date: Fri, 6 Jun 2025 16:30:13 +1000 Subject: [PATCH 25/25] Finish unfinished sentence --- text/3778-proc-macro-in-same-package-as-app.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3778-proc-macro-in-same-package-as-app.md b/text/3778-proc-macro-in-same-package-as-app.md index 58587504ad0..646d706a72f 100644 --- a/text/3778-proc-macro-in-same-package-as-app.md +++ b/text/3778-proc-macro-in-same-package-as-app.md @@ -148,7 +148,7 @@ autoprocmacro = false ## Documentation -There would be a new item listed under "Crates" of the sidebar, for the new crate. This shoul +There would be a new item listed under "Crates" of the sidebar, for the new crate. This should only display "macros" — or whatever the name of the `proc-macro` crate happened to be called — of the current package. # Drawbacks [drawbacks]: #drawbacks