get lage sensor data from ESP ( Json)

[EmileSpecialProducts]

I would like to return sensor data from the ESP in Json format.
Not using the https://arduinojson.org/ as this will not solve my issue.
This works great using this code if the data <64K

 server.on("/OTTemperatures", [](AsyncWebServerRequest *request)
  {
    int f;
   String message;
    message.reserve(64000);
    message = "{";
    message += "\"Temperatures\":[";
    for (f = 0; f < Temperatures->TemperaturesLen; f++)
    {
      message += String(Temperatures->Temperatures[f]);
      if (f != (Temperatures->TemperaturesLen - 1))
        message += ",";
    }
    message += "]}";
   request->send(200, "application/json", message);
}

But if the data is more then 64K byte i have a memory problem ( String only support 64K )

But how can I send blocks of data so that I do not need to have all the Jsondata in memory?

PlatformIO.ini settings

; satble version  3.2.1 version 
platform = https://github.com/pioarduino/platform-espressif32/releases/download/stable/platform-espressif32.zip
lib_deps = 
    https://github.com/ESP32Async/ESPAsyncWebServer.git
    https://github.com/ESP32Async/ESPAsyncTCP.git

Size Limits with Responses

[mathieucarbou]

You can use stream, chunk, SSE, websocket,
Look inside the project for some examples.

[EmileSpecialProducts]

This is how i Resolved it, and works for one request at a time.

static int CurrentSensorData=0;

server.on("/data", HTTP_GET, [](AsyncWebServerRequest *request){
    AsyncWebServerResponse *response = request->beginChunkedResponse("application/json", [](uint8_t *buffer, size_t maxLen, size_t index)-> size_t
    {
      int len =0;
      maxLen-=32;                 // we have padding in here since we check at the end of ever loop and not during
      debugf("/data Maxlen = %d index = %d CurrentSensorData= %d",maxLen,index,CurrentSensorData);
      if(index==0)                // This is our first time in this code.. ASSUME we have enough space for this whole message: Have never seen maxLen too small.
      {
        CurrentSensorData=0;
        len = sprintf((char*)buffer,"{\"Temperatures\":[");
      }
      while((len < maxLen) && CurrentSensorData <4000  )   // while there is room in the buffer and we have not sent it all
      {
        len+=sprintf((char*)buffer+len,"%d",CurrentSensorData);
        CurrentSensorData++;
        if(CurrentSensorData<4000) 
            len+=sprintf((char*)buffer+len,","); 
        else 
            len+=sprintf((char*)buffer+len,"]}");
      }
      debugf(" len = %d\n",len);
      if( len >0) debug((char*)buffer);
      return len;
    });
    request->send(response);
  });

[minou65]

I found a solution for me that works. not sure if it is the best, but it works. I create the webpage in a file, using LittleFS. After creating I show the html by loading the file.

			File file = LittleFS.open("/myhtml.html", "r");
			if (file) {
				AsyncWebServerResponse* response = _request->beginChunkedResponse(
					"text/html",
					[file](uint8_t* buffer, size_t maxLen, size_t index) mutable -> size_t {
						size_t bytesRead = file.read(buffer, maxLen);
						if (bytesRead == 0) {
							file.close();
							LittleFS.remove("/myhtml.html");
						}
						return bytesRead;
					}
				);

				for (const auto& header : _headers) {
					response->addHeader(header.first, header.second);
				}
				_request->send(response);
				_headers.clear();
			}

[EmileSpecialProducts]

Yes,
I have considered this, but the Flash memory has a limited Write cycles generally rated for 100.000 writes.
And it is better to use the request->send(LittleFS, "/myhtml.html", String(), false);
and not the beginChunkedResponse in this case.
And add static before File file = LittleFS.open("/myhtml.html", "r");
But you will still have the issue that 2 requests at the same time will not be handled correctly as you only have one File/object.
When I have a solution, I will post it.

ESP-LittleFS-Async-Web-Server

Or for SD card <= 32GB portable-Async-disk-drive and this is working

You can find my code portable-Async-disk-driveEx this is under construction as this has the same problem regarding the 2 requests at the same time will not be handled correctly using FatEx.