Top-level static files such as robots.txt #4404

jongiddy · 2025-07-11T06:27:03Z

jongiddy
Jul 11, 2025

Is there any way to add a static asset at the same level as the index.html file in a bundle? For example, a robots.txt file that must keep the same name, without any hash, and must be served from the top-level path /robots.txt.

dustmoo · 2025-07-31T19:10:44Z

dustmoo
Jul 31, 2025

Dropping this here for those that need a robots.txt file @jongiddy. While not simple, you can serve robots.txt programmatically by using the axum integration. This allows you to add specific routes that take priority over the Dioxus renderer.

A common pitfall I ran into is ensuring your static assets (like CSS) are configured correctly for the server build (global instead of web target). The key is to use a Dioxus.toml file to declare your asset directory at the application level. Here is an example:

1. Configure `Dioxus.toml`

First, ensure your assets are configured at the application level. This tells the build tool what to copy for the server to use.

# Dioxus.toml
[application]
name = "my-dioxus-app"

# This `asset_dir` is read by the build tool and copies the `assets`
# folder into the final build output for ALL platforms, including the server.
asset_dir = "assets"

[web.app]
# You can still have web-specific settings
title = "My Dioxus App"
# You can also add a <head> section here to link CSS
head = '''
    <link rel="stylesheet" href="/assets/style.css">
'''

2. Configure `Cargo.toml`

Next, set up your dependencies with optional features for the server. This example includes common utilities like dotenv and dioxus_logger.

# Cargo.toml
[package]
name = "dioxus-robots-example"
version = "0.1.0"
edition = "2021"

[dependencies]
dioxus = { version = "0.6", features = ["fullstack", "logger"] }

# Server-only dependencies are marked as optional
axum = { version = "0.7", optional = true }
tokio = { version = "1", features = ["full"], optional = true }
dioxus-fullstack = { version = "0.6.0", features = ["axum"], optional = true }
dotenv = { version = "0.15.0", optional = true }
dioxus_logger = { version = "0.6.0", optional = true }
tracing = { version = "0.1", optional = true }

[features]
default = ["web", "server"]
web = ["dioxus/web"]
server = [
    "dioxus/server",
    "dep:axum",
    "dep:tokio",
    "dep:dioxus-fullstack",
    "dep:dotenv",
    "dep:dioxus_logger",
    "dep:tracing"
]

3. Configure `src/main.rs`

Finally, set up your main.rs to launch the custom server.

use dioxus::prelude::*;

fn App() -> Element {
    rsx! { h1 { "Hello, World!" } }
}

fn main() {
    #[cfg(feature = "web")]
    dioxus::launch(App);

    #[cfg(feature = "server")]
    {
        tokio::runtime::Runtime::new()
            .unwrap()
            .block_on(async move {
                launch_server().await;
            });
    }
}

#[cfg(feature = "server")]
async fn launch_server() {
    use axum::routing::get;
    use axum::{body::Body, http::header, response::{IntoResponse, Response}, Router};
    use dioxus_fullstack::prelude::*;
    use dotenv::dotenv;
    use std::net::{IpAddr, Ipv4Addr, SocketAddr};

    // Load .env file and initialize logger
    dotenv().ok();
    dioxus_logger::init(tracing::Level::DEBUG).expect("failed to init logger");

    // Get the address to run the server on
    let ip = dioxus::cli_config::server_ip().unwrap_or_else(|| IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)));
    let port = dioxus::cli_config::server_port().unwrap_or(8080);
    let address = SocketAddr::new(ip, port);

    // A more robust handler for robots.txt that sets the Content-Type header
    let robots_txt_handler = || async {
        let content = "User-agent: *\nAllow
: /";
        Response::builder()
            .header(header::CONTENT_TYPE, "text/plain")
            .body(Body::from(content))
            .unwrap()
    };

    // Build the Axum router
    let router = Router::new()
        // Your custom route takes precedence
        .route("/robots.txt", get(robots_txt_handler))
        // Let Dioxus handle everything else (app rendering, static assets, etc.)
        .serve_dioxus_application(ServeConfigBuilder::new().build().unwrap(), App);

    // Run the server
    let listener = tokio::net::TcpListener::bind(address).await.unwrap();
    tracing::info!("🤖 Server starting on http://{}", listener.local_addr().unwrap());
    axum::serve(listener, router).await.unwrap();
}

// Your App component, shared between client and server
//#[component]
//fn App() -> Element { ...

Now when you run dx serve, your custom route for robots.txt will be served correctly, and because the Dioxus.toml is configured properly, the .serve_dioxus_application() method will handle all of your static assets as well.

(Optional) Nginx Reverse Proxy

If you prefer to handle static files with a dedicated web server, you can place Nginx in front of your Dioxus application. In this setup, Nginx serves static files like robots.txt and your CSS directly, and forwards all other requests to your Axum server.

Here is a simple Nginx configuration for this approach:

# /etc/nginx/sites-available/your-app.conf

server {
    listen 80;
    server_name your-domain.com;
    root /path/to/your/project/dist; # Point to the Dioxus build output

    # 1. Serve robots.txt directly
    location = /robots.txt {
        try_files $uri =404;
        add_header Content-Type text/plain;
    }

    # 2. Serve all other static assets
    # Nginx will look for files in /path/to/your/project/dist/assets
    location /assets/ {
        try_files $uri =404;
    }

    # 3. Proxy all other requests to the Dioxus/Axum server
    location / {
        proxy_pass [http://127.0.0.1:8080](http://127.0.0.1:8080); # Assuming your app runs on port 8080
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Top-level static files such as robots.txt #4404

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Uh oh!

Top-level static files such as robots.txt #4404

Uh oh!

jongiddy Jul 11, 2025

Replies: 1 comment

Uh oh!

Uh oh!

dustmoo Jul 31, 2025

1. Configure Dioxus.toml

2. Configure Cargo.toml

3. Configure src/main.rs

(Optional) Nginx Reverse Proxy

jongiddy
Jul 11, 2025

dustmoo
Jul 31, 2025

1. Configure `Dioxus.toml`

2. Configure `Cargo.toml`

3. Configure `src/main.rs`