fix(docs): updating docs structure

⚠️ this PR builds off of the changes in #16442, so merge that one first

This PR makes several edits to the docs. The main impetus behind the changes being made is to make a more linear learning journey for developers as they move down the sidebar.

- moves the guides section above the tutorials
- flattens the pages inside the "Developer smart contracts" section
- makes the "developing smart contracts" section a more linear flow for how a developer should be moving through this content
- moves some of the technical explanations to the concepts section
- adds a several pages to the Guides section including:
  - project sturcturing
  - data types
  - custom types
  - defining functions (initializer page moved into here)
 - removes some explainers and reference about storage from the references section into the guides section

closes: AztecProtocol/dev-rel#598

Co-authored-by: Alex Gherghisan <[email protected]>
Co-authored-by: Charlie Lye <[email protected]>
Co-authored-by: Facundo <[email protected]>
Co-authored-by: IlyasRidhuan <[email protected]>
Co-authored-by: Jonathan Hao <[email protected]>
Co-authored-by: Josh Crites <[email protected]>
Co-authored-by: José Pedro Sousa <[email protected]>
Co-authored-by: LHerskind <[email protected]>
Co-authored-by: Lasse Herskind <[email protected]>
Co-authored-by: Raju Krishnamoorthy <[email protected]>
Co-authored-by: fcarreiro <[email protected]>
Co-authored-by: federicobarbacovi <[email protected]>
Co-authored-by: jeanmon <[email protected]>
Co-authored-by: maramihali <[email protected]>
Co-authored-by: sergei iakovenko <[email protected]>
Co-authored-by: sirasistant <[email protected]>
Co-authored-by: thunkar <[email protected]>

Author: 16 people

docs/docs-words.txt

@@ -327,6 +327,7 @@ undeflow
 Undercollateralized
 underconstrained
 unleaked
+unlinkability
 unncessary
 unreverted
 unrleated

docs/docs/aztec/concepts/accounts/keys.md

@@ -106,7 +106,7 @@ When it comes to storing the signing key in a private note, there are several de
 
 #### Using Delayed Public Mutable state
 
-By [Delayed Public Mutable](../../../developers/reference/smart_contract_reference/storage/delayed_public_mutable.md#delayedpublicmutable) we mean privately readable publicly mutable state.
+By [Delayed Public Mutable](../../../developers/guides/smart_contracts/storage_types.md#delayed-public-mutable) we mean privately readable publicly mutable state.
 
 To make public state accessible privately, there is a delay window in public state updates. One needs this window to be able to generate proofs client-side. This approach would not generate additional nullifiers and commitments for each transaction while allowing the user to rotate their key. However, this causes every transaction to now have a time-to-live determined by the frequency of the delayed mutable state, as well as imposing restrictions on how fast keys can be rotated due to minimum delays.
 

docs/docs/aztec/concepts/advanced/authwit.md

@@ -141,4 +141,4 @@ We don't need to limit ourselves to the `transfer` function, we can use the same
 
 ### Next Steps
 
-Check out the [developer documentation](../../../developers/guides/smart_contracts/writing_contracts/authwit.md) to see how to implement this in your own contracts.
+Check out the [developer documentation](../../../developers/guides/smart_contracts/authwit.md) to see how to implement this in your own contracts.

docs/docs/aztec/concepts/advanced/storage/storage_slots.md

@@ -5,8 +5,6 @@ sidebar_position: 1
 description: Understand how storage slots work in Aztec for both public and private state, including siloing mechanisms and note hash commitments.
 ---
 
-# Storage Slots
-
 ## Public State Slots
 
 As mentioned in [State Model](../../storage/state_model.md), Aztec public state behaves similarly to public state on Ethereum from the point of view of the developer. Behind the scenes however, the storage is managed differently. As mentioned, public state has just one large sparse tree in Aztec - so we silo slots of public data by hashing it together with its contract address.
@@ -58,4 +56,60 @@ By doing this address-siloing at the kernel circuit we _force_ the inserted comm
 To ensure that nullifiers don't collide across contracts we also force this contract siloing at the kernel level.
 :::
 
-For an example of this see [developer documentation on storage](../../../../developers/reference/smart_contract_reference/storage/index.md).
+## Example
+
+In this section we will go into more detail and walk through an entire example of how storage slots are computed for private state to improve our storage slot intuition. Recall, that storage slots in the private domain is just a logical construct, and are not "actually" used for lookups, but rather just as a value to constrain against.
+
+For the case of the example, we will look at what is inserted into the note hashes tree when adding a note in the Token contract. Specifically, we are looking at the last part of the `transfer` function:
+
+#include_code increase_private_balance noir-projects/noir-contracts/contracts/app/token_contract/src/main.nr rust
+
+This function is creating a new note and inserting it into the balance set of the recipient `to`. Recall that to ensure privacy, only the note hash is really inserted into the note hashes tree. To share the contents of the note with `to` the contract can emit an encrypted log (which this one does), or it can require an out-of-band data transfer sharing the information. Below, we will walk through the steps of how the note hash is computed and inserted into the tree. For this, we don't care about the encrypted log, so we are going to ignore that part of the function call for now.
+
+Outlining it in more detail below as a sequence diagram, we can see how the calls make their way down the stack.
+In the end a siloed note hash is computed in the kernel.
+
+:::info
+Some of the syntax below is a little butchered to make it easier to follow variables without the full code.
+:::
+
+```mermaid
+sequenceDiagram
+    alt Call
+    Token->>BalanceMap: Map::new(map_slot);
+    Token->>Token: to_bal = storage.balances.at(to)
+    Token->>BalanceMap: BalanceMap.at(to)
+    BalanceMap->>BalanceMap: derived_slot = H(map_slot, to)
+    BalanceMap->>BalanceSet: BalanceSet::new(to, derived_slot)
+    Token->>BalanceSet: to_bal.add(amount)
+    BalanceSet->>BalanceSet: note = UintNote::new(amount, to)
+    BalanceSet->>Set: insert(note)
+    Set->>LifeCycle: create_note(derived_slot, note)
+    LifeCycle->>LifeCycle: note.header = NoteHeader { contract_address, <br> storage_slot: derived_slot, nonce: 0, note_hash_counter }
+    UintPartialNotePrivateContent->>UintNote: note_hash = compute_partial_commitment(storage_slot).x
+    LifeCycle->>Context: push_note_hash(note_hash)
+    end
+    Context->>Kernel: unique_note_hash = H(nonce, note_hash)
+    Context->>Kernel: siloed_note_hash = H(contract_address, unique_note_hash)
+```
+
+Notice the `siloed_note_hash` at the very end. It's a hash that will be inserted into the note hashes tree. To clarify what this really is, we "unroll" the values to their simplest components. This gives us a better idea around what is actually inserted into the tree.
+
+```rust
+siloed_note_hash = H(contract_address, unique_note_hash)
+siloed_note_hash = H(contract_address, H(nonce, note_hash))
+siloed_note_hash = H(contract_address, H(H(tx_hash, note_index_in_tx), note_hash))
+siloed_note_hash = H(contract_address, H(H(tx_hash, note_index_in_tx), MSM([G_amt, G_to, G_rand, G_slot], [amount, to, randomness, derived_slot]).x))
+```
+
+MSM is a multi scalar multiplication on a grumpkin curve and G\_\* values are generators.
+
+And `to` is the actor who receives the note, `amount` of the note and `randomness` is the randomness used to make the note hiding. Without the `randomness` the note could just as well be plaintext (computational cost of a preimage attack would be trivial in such a case).
+
+:::info
+Beware that this hash computation is what the aztec.nr library is doing, and not strictly required by the network (only the kernel computation is).
+:::
+
+With this note structure, the contract can require that only notes sitting at specific storage slots can be used by specific operations, e.g., if transferring funds from `from` to `to`, the notes to destroy should be linked to `H(map_slot, from)` and the new notes (except the change-note) should be linked to `H(map_slot, to)`.
+
+That way, we can have logical storage slots, without them really existing. This means that knowing the storage slot for a note is not enough to actually figure out what is in there (whereas it would be for looking up public state).

docs/docs/aztec/concepts/call_types.md

@@ -149,7 +149,7 @@ aztec = { git = "https://github.com/AztecProtocol/aztec-packages/", tag = "#incl
 
 Even with the router contract achieving good privacy is hard.
 For example, if the value being checked against is unique and stored in the contract's public storage, it's then simple to find private transactions that are using that value in the enqueued public reads, and therefore link them to this contract.
-For this reason it is encouraged to try to avoid public function calls and instead privately read [Shared State](../../developers/reference/smart_contract_reference/storage/delayed_public_mutable.md) when possible.
+For this reason it is encouraged to try to avoid public function calls and instead privately read [Shared State](../../developers/guides/smart_contracts/storage_types.md#delayed-public-mutable) when possible.
 
 ### Public Execution
 

docs/docs/aztec/concepts/pxe/index.md

@@ -106,4 +106,4 @@ Oracles are pieces of data that are injected into a smart contract function from
 To learn how to develop on top of the PXE, refer to these guides:
 
 - [Run more than one PXE on your local machine](../../../developers/guides/local_env/sandbox.md#running-multiple-pxes-in-the-sandbox)
-- [Use in-built oracles including oracles for arbitrary data](../../../developers/guides/smart_contracts/writing_contracts/how_to_use_capsules.md)
+- [Use in-built oracles including oracles for arbitrary data](../../../developers/guides/smart_contracts/advanced/how_to_use_capsules.md)

docs/docs/aztec/concepts/storage/notes.md

@@ -43,4 +43,145 @@ When using the Aztec protocol, users may not be aware of the specific notes that
 
 This is accomplished through the smart contract library, Aztec.nr, which abstracts notes by allowing developers to specify custom note types. This means they can specify how notes are interacted with, nullified, transferred, and displayed. Aztec.nr also helps users discover all of the notes that have been encrypted to their account and posted to the chain, known as [note discovery](../advanced/storage/note_discovery.md).
 
-To understand note abstraction in Aztec.nr, you can read the [Build section](../../../developers/guides/smart_contracts/writing_contracts/notes/index.md).
+## Technical details
+
+### Some context
+
+- Public functions and storage work much like other blockchains in terms of having dedicated storage slots and being publicly visible
+- Private functions are executed locally with proofs generated for sound execution, and commitments to private variable updates are stored using append-only trees
+- "Note" types are part of Aztec.nr, a framework that facilitates use of Aztec's different storage trees to achieve things such as private variables
+
+This page will focus on how private variables are implemented with Notes and storage trees.
+
+#### Side-note about execution
+
+Under the hood, the Aztec protocol handles some important details around public and private function calls. Calls between them are asynchronous due to different execution contexts (local execution vs. node execution).
+A detailed explanation of the transaction lifecycle can be found [here](../transactions.md#simple-example-of-the-private-transaction-lifecycle).
+
+## Private state variables in Aztec
+
+State variables in an Aztec contract are defined inside a struct specifically named `Storage`, and must satisfy the [Note Interface (GitHub link)](https://github.com/AztecProtocol/aztec-packages/tree/#include_aztec_version/noir-projects/aztec-nr/aztec/src/note/note_interface.nr) and contain a [Note header (GitHub link)](https://github.com/AztecProtocol/aztec-packages/tree/#include_aztec_version/noir-projects/aztec-nr/aztec/src/note/note_header.nr).
+
+The Note header struct contains the contract address which the value is effectively siloed to, a nonce to ensure unique Note hashes, and a storage "slot" (or ID) to associate multiple notes.
+
+A couple of things to unpack here:
+
+#### Storage "slot"
+
+Storage slots are more literal for public storage, a place where a value is stored. For private storage, a storage slot is logical (more [here](../advanced/storage/storage_slots.md)).
+
+#### Silos
+
+The address of the contract is included in a Note's data to ensure that different contracts don't arrive at the same hash with an identical variable. This is handled in the protocol's execution.
+
+### Note types
+
+There is more than one Note type, such as the `PrivateSet` type is used for private variables. There are also `PrivateMutable` and `PrivateImmutable` types.
+
+Furthermore, notes can be completely custom types, storing any value or set of values that are desired by an application.
+
+### Initialization
+
+Private state variables are stored locally when the contract is created. Depending on the application, values may be privately shared by the creator with others via encrypted logs onchain.
+A hash of a note is stored in the append-only note hash tree on the network so as to prove existence of the current state of the note in a privacy preserving way.
+
+#### Note Hash Tree
+
+By virtue of being append only, notes are not edited. If two transactions amend a private value, multiple notes will be inserted into the tree to the note hash tree and the nullifier tree. The header will contain the same logical storage slot.
+
+### Reading Notes
+
+:::info
+
+Only those with appropriate keys/information will be able to successfully read private values that they have permission to. Notes can be read outside of a transaction or "off-chain" with no changes to data structures on-chain.
+
+:::
+
+When a note is read in a transaction, a subsequent read from another transaction of the same note would reveal a link between the two. So to preserve privacy, notes that are read in a transaction are said to be "consumed" (defined below), and new note(s) are then created with a unique hash.
+
+With type `PrviateSet`, a private variable's value is interpreted as the sum of values of notes with the same logical storage slot.
+
+Consuming, deleting, or otherwise "nullifying" a note is NOT done by deleting the Note hash; this would leak information. Rather a nullifier is created deterministically linked to the value. This nullifier is inserted into another the nullifier storage tree.
+
+When reading a value, the local private execution checks that its notes (of the corresponding storage slot/ID) have not been nullified.
+
+### Updating
+
+:::note
+Only those with appropriate keys/information will be able to successfully nullify a value that they have permission to.
+:::
+
+To update a value, its previous note hash(es) are nullified. The new note value is updated in the user's private execution environment (PXE), and the updated note hash inserted into the note hash tree.
+
+## Supplementary components
+
+Some optional background resources on notes can be found here:
+
+- [High level network architecture](../../index.md), specifically the Private Execution Environment
+- [Transaction lifecycle (simple diagram)](../transactions.md#simple-example-of-the-private-transaction-lifecycle)
+- [Public and Private state](./state_model.md)
+
+Notes touch several core components of the protocol, but we will focus on a the essentials first.
+
+### Some code context
+
+The way Aztec benefits from the Noir language is via three important components:
+
+- `Aztec.nr` - a Noir framework enabling contracts on Aztec, written in Noir. Includes useful Note implementations
+- `noir contracts` - example Aztec contracts
+- `noir-protocol-circuits` - a crate containing essential circuits for the protocol (public circuits and private wrappers)
+
+A lot of what we will look at will be in [aztec-nr/aztec/src/note (GitHub link)](https://github.com/AztecProtocol/aztec-packages/tree/#include_aztec_version/noir-projects/aztec-nr/aztec/src/note), specifically the lifecycle and note interface.
+
+Looking at the noir circuits in these components, you will see references to the distinction between public/private execution and state.
+
+### Lifecycle functions
+
+Inside the [lifecycle (GitHub link)](https://github.com/AztecProtocol/aztec-packages/tree/#include_aztec_version/noir-projects/aztec-nr/aztec/src/note/lifecycle.nr) circuits we see the functions to create and destroy a note, implemented as insertions of note hashes and nullifiers respectively. This is helpful for regular private variables.
+
+We also see a function to create a note hash from the public context, a way of creating a private variable from a public call (run in the sequencer). This could be used in application contracts to give private digital assets to users.
+
+### Note Interface functions
+
+To see a [note_interface (GitHub link)](https://github.com/AztecProtocol/aztec-packages/tree/#include_aztec_version/noir-projects/aztec-nr/aztec/src/note/note_interface.nr) implementation, we will look at a simple [ValueNote GitHub link](https://github.com/AztecProtocol/aztec-packages/tree/#include_aztec_version/noir-projects/aztec-nr/value-note/src/value_note.nr).
+
+The interface is required to work within an Aztec contract's storage, and a ValueNote is a specific type of note to hold a number (as a `Field`).
+
+#### Computing hashes and nullifiers
+
+A few key functions in the note interface are around computing the note hash and nullifier, with logic to get/use secret keys from the private context.
+
+In the ValueNote implementation you'll notice that it uses the `pedersen_hash` function. This is currently required by the protocol, but may be updated to another hashing function, like poseidon.
+
+As a convenience, the outer [note/utils.nr (GitHub link)](https://github.com/AztecProtocol/aztec-packages/tree/#include_aztec_version/noir-projects/aztec-nr/aztec/src/note/utils.nr) contains implementations of functions that may be needed in Aztec contracts, for example computing note hashes.
+
+#### Serialization and deserialization
+
+Serialization/deserialization of content is used to convert between the Note's variables and a generic array of Field elements. The Field type is understood and used by lower level crypographic libraries.
+This is analogous to the encoding/decoding between variables and bytes in solidity.
+
+For example in ValueNote, the `serialize_content` function simply returns: the value, nullifying public key hash (as a field) and the note randomness; as an array of Field elements.
+
+### Value as a sum of Notes
+
+We recall that multiple notes are associated with a "slot" (or ID), and so the value of a numerical note (like ValueNote) is the sum of each note's value.
+The helper function in [balance_utils (GitHub link)](https://github.com/AztecProtocol/aztec-packages/blob/#include_/noir-projects/aztec-nr/value-note/src/balance_utils.nr) implements this logic taking a `PrivateSet` of `ValueNotes`.
+
+A couple of things worth clarifying:
+
+- A `PrivateSet` takes a Generic type, specified here as `ValueNote`, but can be any `Note` type (for all notes in the set)
+- A `PrivateSet` of notes also specifies _the_ slot of all Notes that it holds
+
+### Example - Notes in action
+
+The Aztec.nr framework includes examples of high-level states [easy_private_uint (GitHub link)](https://github.com/AztecProtocol/aztec-packages/tree/#include_aztec_version/noir-projects/aztec-nr/easy-private-state/src/easy_private_uint.nr) for use in contracts.
+
+The struct (`EasyPrivateUint`) contains a Context, Set of ValueNotes, and storage_slot (used when setting the Set).
+
+Notice how the `add` function shows the simplicity of appending a new note to all existing ones. On the other hand, `sub` (subtraction), needs to first add up all existing values (consuming them in the process), and then insert a single new value of the difference between the sum and parameter.
+
+---
+
+### References
+
+- ["Stable" state variable (GitHub link)](https://github.com/AztecProtocol/aztec-packages/issues/4130)

docs/docs/aztec/concepts/storage/state_model.md

@@ -22,7 +22,3 @@ Private state is represented in an append-only database since updating a record
 The act of "deleting" a private state variable can be represented by adding an associated nullifier to a nullifier set. The nullifier is generated such that, without knowing the decryption key of the owner, an observer cannot link a state record with a nullifier.
 
 Modification of state variables can be emulated by nullifying the state record and creating a new record to represent the variable. Private state has an intrinsic UTXO structure.
-
-## Further reading
-
-Read more about how to leverage the Aztec state model in Aztec contracts [here](../../../developers/reference/smart_contract_reference/storage/index.md).

docs/docs/aztec/smart_contracts/contract_creation.md

@@ -61,7 +61,7 @@ Aztec makes an important distinction between initialization and public deploymen
 
 ### Initialization
 
-Contract constructors are not enshrined in the protocol, but handled at the application circuit level. Constructors are methods used for initializing a contract, either private or public, and contract classes may declare more than a single constructor. They can be declared by the `#[initializer]` macro. You can read more about how to use them on the [Defining Initializer Functions](../../developers/guides/smart_contracts/writing_contracts/initializers.md) page.
+Contract constructors are not enshrined in the protocol, but handled at the application circuit level. Constructors are methods used for initializing a contract, either private or public, and contract classes may declare more than a single constructor. They can be declared by the `#[initializer]` macro. You can read more about how to use them on the [Defining Initializer Functions](../../developers/guides/smart_contracts/define_functions.md#initializer-functions) page.
 
 A contract must ensure: