Sitemap generation error on browser

[okfmohamed]

To Reproduce

1. Visit the site: https://mysite.com/sitemap.xml

2. This error will be shown on the browser:

XML Parsing Error: no root element found
Location: https://mysite.com/sitemap.xml
Line Number 1, Column 1:

3. Visit the same site: again after waiting for few minutes

4. This error will be shown in the browser:

error

This page isn’t working www.mysite.com took too long to respond. HTTP ERROR 504

5. Visit the same site: again (third time) but only after waiting longer (few more minutes)

6. Now the sitemap will be output correctly on the browser window

Expected behavior

The sitemap file is not always output correctly on the browser window and many times the browser show different errors

Describe the bug

Not sure why this keeps happening but this error is not consistently replicated while probably it has to do with the way the module generates the sitemap keeping in mind that all default configuration settings were used. Also not sure if there is cache setting added by the default settings. No manual command line tasks or cron jobs were used to generate the XML file

Details

Server Operating System:
Ubuntu/nginx

Version of Node.js:
8.9.3

Version of Apostrophe
2.94.1

Version of Apostrophe site map
2.5.0

Additional context:

These errors even though are different each time I visit the browser, were always caused when the browser keeps waiting for the XML sitemap to be generated and after few seconds the browser starts to show all these errors.

This is not consistent and can not always be replicated because the XML sitemap is output correctly on some occasions and on many others is not

Screenshots

[boutell]

Hi, to reproduce this we'll need a sample projects and steps to duplicate the issue. Thanks!

…

On Wed, Feb 3, 2021 at 8:02 PM Omar ***@***.***> wrote: To Reproduce 1. Visit the site: https://mysite.com/sitemap.xml 1. This error will be shown on the browser: XML Parsing Error: no root element found Location: https://mysite.com/sitemap.xml Line Number 1, Column 1: 1. Visit the same site: again after waiting for few minutes 1. This error will be shown in the browser: /error 1. Visit the same site: again but after waiting longer maybe for few more minutes (third time) 1. Now the sitemap is output correctly on the browser window Expected behavior The sitemap file is not always output correctly on the browser window and many times the browser show different errors Describe the bug Not sure why this keeps happening but this error is not consistently replicated while probably has to do with the way the module generates the sitemap keeping in mind that all default configuration settings was used. Also not sure if there is cache setting enforced with the default settings but no manual command line tasks or cron jobs were used to generates the XML file Details *Server Operating System:* Ubuntu/nginx *Version of Node.js:* 8.9.3 *Version of Apostrophe* 2.94.1 *Version of Apostrophe site map* 2.5.0 *Additional context:* These errors even though are different each time I visit the browser, were always caused when the browser keeps waiting for the XML sitemap to be generated and after few minutes the browser starts to show all these errors. This is not consistent and can not always can be replicated because the XML sitemap is output correctly on some occasions and on many others is not *Screenshots* — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#43>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27IUJ32P6RO7HAG2IWDS5HW3RANCNFSM4XB34ISQ> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[okfmohamed]

@boutell

I was able to replicate the issue and I know what happened. Basically, the default setting of 100 for piecesPerBatch were taking long time to execute on the back end server as AWS load balancer kept waiting and waiting until the default idle timeout value of 60 seconds was reached. When I increased the piecesPerBatch option to 500 and to 1000 the code took less time to finish running and output the sitemap this time with no errors because the ELB did not terminate the connection since the default 60 seconds idle timeout was never reached. These are my concerns though:

1. The back end database is about 1.5 GB and I am wondering how will this scale up with huge databases that has Terra bytes of data? Obviously, we could just increase the CPU and the server configuration I/O but that might not always be feasible while AWS do suggest another approach to resolve these HTTP 504 errors:

2. Not sure if this edge case is already handled in the code base, This would be to sent 1 bytes of data each time before the idle timeout expires (so every 60 seconds/ 30 seconds intervals depending on each web host) and that would make sure the ELB will not terminate the connection to the client due to inactivity as was the case when the request was processed by the ec2 instances as the module was generating the sitemap and cache it in the database. This AWS excerpt regarding HTTP 504 errors shown on this ticket:

HTTP 504 Gateway error:
To ensure that lengthy operations such as file uploads have time to complete, send at least 1 byte of data before each idle timeout period elapses, and increase the length of the idle timeout period as needed.

Another question: I want to change the default cache period instead of 1 hour. Is cacheLifetime setting in the code interpreted in seconds or minutes: Example: cacheLifetime: 1440 (Is this equivalent to 1 day = 60 minutes * 24 hours)?

Screen shot below:

Thank you for your help!

[boutell]

Thanks for the thoughtful research! The easiest way to avoid this problem is to use a scheduled task, such as a cron job, that runs often enough that the cache can never expire for actual web requests. This is suggested in the documentation. The cache lifetime is specified in seconds. I do wonder if we can rely on that one-byte-per-second trick in some other situations, but my instincts tell me it's rude somehow. (: Or that not all proxies will consider it good enough.

…

On Sun, Feb 7, 2021 at 12:36 AM Omar ***@***.***> wrote: @boutell <https://github.com/boutell> I was able to replicate the issue and I know what happened. Basically, the default setting of 100 for piecesPerBatch were taking long time to execute on the back end server as AWS load balancer kept waiting and waiting until the default idle timeout value of 60 seconds was reached. When I increased the piecesPerBatch option to 500 and to 1000 the code took less time to finish running and output the sitemap this time with no errors because the ELB did not terminate the connection since the default 60 seconds idle time was never reached. These are my concerns though: 1. The back end database is about 1.5 GB and I am wondering how will this scale up with huge databases that has Terra bytes of data? Obviously, we could just increase the CPU and the server configuration I/O but that might not always be feasible while AWS do suggest another approach to resolve these HTTP 504 errors: 2. Though not sure if this edge case is already handled in the code base is to sent *1 bytes* of data each time before the idle time out expires (so every 60 seconds/ 30 seconds intervals depending on each web host) and that would make sure the ELB will not terminate the connection to the client due to inactivity as was the case when the request was processed by the ec2 instances as the module was generating the sitemap and cache it in the database. This AWS excerpt regarding HTTP 504 errors shown on this ticket: HTTP 504 Gateway error: To ensure that lengthy operations such as file uploads have time to complete, send at least 1 byte of data before each idle timeout period elapses, and increase the length of the idle timeout period as needed. Another question: I want to change the default cache period OF 1 hour. Is cacheLifetime setting in the code interpreted as seconds or minutes: Example: cacheLifetime: 1440 (Is this equivalent to 1 day = 60 minutes * 24 hours)? Screen shot below: [image: 504-error-chrome] <https://user-images.githubusercontent.com/39920614/107137377-e1c66680-68d9-11eb-8dbb-9db09c6d1b62.png> Thank you for your help! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27I6WMTVYB3SBLJ6NE3S5YREBANCNFSM4XB34ISQ> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[okfmohamed]

@boutell

Thank you for your reply.

As per AWS Docs, and since they recommended using this to address 504 errors, it can be very well considered as fix to address this issue unless you have any better ideas?

[boutell]

The best way to fix this issue is to use a scheduled task for this, and never let the cache expire. For sites at scale in production you never want this to happen "while you wait."

…

On Mon, Feb 8, 2021 at 3:49 PM Omar ***@***.***> wrote: @boutell <https://github.com/boutell> Thank you for your reply. As per AWS Docs, and since they recommended using this to address 504 errors, it can be very well considered as fix to address this issue unless you have any better ideas? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27O6WMOJ63XEF563O6LS6BE55ANCNFSM4XB34ISQ> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[boutell]

Please see: https://github.com/apostrophecms/apostrophe-site-map#generating-the-sitemap-ahead-of-time

…

On Mon, Feb 8, 2021 at 4:09 PM Tom Boutell ***@***.***> wrote: The best way to fix this issue is to use a scheduled task for this, and never let the cache expire. For sites at scale in production you never want this to happen "while you wait." On Mon, Feb 8, 2021 at 3:49 PM Omar ***@***.***> wrote: > @boutell <https://github.com/boutell> > > Thank you for your reply. > > As per AWS Docs, and since they recommended using this to address 504 > errors, it can be very well considered as fix to address this issue unless > you have any better ideas? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#43 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAH27O6WMOJ63XEF563O6LS6BE55ANCNFSM4XB34ISQ> > . > -- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[okfmohamed]

Yes I already did and thanks for the info

[okfmohamed]

I was just hoping that you would consider AWS suggestion as well.

Thank you for your help

[okfmohamed]

@boutell

Even if we run cron job to generate the sitemap dynamically without letting Google wait for the sitemap, that doesn't necessarily mean the very first cron job will run with no errors for super large databases because AWS elastic load balancer will still return HTTP 504 network time out error once the idle time out expires before the task finishes running. Yes, it is remedy but that will not always work without network timeout errors

[boutell]

That's an interesting point about the very first run.

…

On Mon, Feb 8, 2021 at 11:23 PM Omar ***@***.***> wrote: @boutell <https://github.com/boutell> Even if we run cron job to generate the sitemap dynamically without letting Google wait for the sitemap, that doesn't necessarily mean the very first cron job will run with no errors for super large databases because AWS elastic load balancer will still return HTTP 504 network time out error once the idle time out expires before the task finishes running. Yes,it is remedy, but that will not always work without network timeout errors — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27NJYNGHJGMTNA77NJ3S6C2EDANCNFSM4XB34ISQ> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his