Conversation
|
According to https://chameleon.readthedocs.io/en/latest/configuration.html either |
|
I am working in a sandbox with The errors are all the same it seems. There are supposedly indentation errors towards the top of the file where the import statements are. But when I look at the compiled source they look fine, see example below. The only oddity I can see is that every one of the supposedly faulty source files has a whole bunch of identical Example error: The top of the mentioned file # -*- coding: utf-8 -*-
# template: <string>
#
__filename = '<string>'
__tokens = {31: ('modules/ZTUtils', 2, 24), 48: (' python:ztu.Batch(range(10), 5', 2, 41), 103: ('b', 3, 21), 146: ('n', 4, 39), 188: ('b/next', 6, 21), 236: ('n', 7, 39)}
from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
from sys import exc_info as _exc_info
from Products.PageTemplates.engine import _compile_zt_expr as __compile_zt_expr
from Products.PageTemplates.engine import _C2ZContextWrapper as __C2ZContextWrapper
_static_4589148944 = {}
...Partial diff comparing Python output between Chameleon 4.4.4 and 4.6.0 (diff between 4.4.4 and 4.5.0 as the first failing version is similar): --- chameleon444/38eaa4e4a325ebb3b85247942ea63989.py 2025-02-21 09:04:11
+++ chameleon460/167832ff5f17ce388c229d45458f835d.py 2025-02-21 09:03:36
@@ -4,15 +4,31 @@
__filename = '<string>'
__tokens = {31: ('modules/ZTUtils', 2, 24), 48: (' python:ztu.Batch(range(10), 5', 2, 41), 103: ('b', 3, 21), 146: ('n', 4, 39), 188: ('b/next', 6, 21), 236: ('n', 7, 39)}
-
-from Products.PageTemplates.expression import BoboAwareZopeTraverse as _BoboAwareZopeTraverse
-from sys import exc_info as _exc_info
from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
-from AccessControl.cAccessControl import guarded_getattr as _guarded_getattr
-
-_static_4432424208 = _BoboAwareZopeTraverse()
-_static_4419559312 = {}
-
+from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
+from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
+from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
+from sys import exc_info as _exc_info
+from Products.PageTemplates.engine import _compile_zt_expr as __compile_zt_expr
+from Products.PageTemplates.engine import _C2ZContextWrapper as __C2ZContextWrapper
+_static_4416692816 = {} |
|
One more data point: When I unset |
|
Another data point about the "new" traceback: Under Chameleon 4.6.0 the expression Under Chameleon 4.4.4 this expression looks like I don't know why that expression is so different between Chameleon 4.4.4 and Chameleon 4.(5|6).0. |
mauritsvanrees
force-pushed
the
chameleon-460
branch
from
26fd999 to
14a58ff
Compare
|
I have rebased on master and force pushed, so we have a clean history again. Still the same problems. |
|
I am starting to wonder if we can disable these failing tests, or most of them. In the Plone 6.2 core development buildout locally I added a checkout of Zope and I use its versions, plus Chameleon 4.6.0, and all tests pass. (Well, a few fail, but that is true with Chameleon 4.4.4 as well, and the test failures do not seem to have anything to do with page templates.) |
@dataflake I tried to reproduce what you found, also using CHAMELEON_CACHE to view the generated .py files. However in my case I do see an actual indentation problem on line 8: In this case the template source code was:
|
|
Indentation or not, any Chameleon version >= 4.5 produces different Python representations from 4.4 and that Python representation doesn't work. I do not know why this is and I am not a Chameleon expert. The one person who has picked up these issues in the past is no longer participating. I am really annoyed that no one on the Chameleon side seems to care. |
|
@dataflake it looks like there was a significant change to how Chameleon generates code from an AST in Chameleon 4.5.0, to use more stdlib code and be more resilient to changes in CPython. I started digging into why that causes this issue today, but ran out of time and will have to come back to it. Then I'll update the upstream issue. |
With all due respect, "clicking around in Plone" doesn't really help large production systems, be it on Plone or Zope, that have hundreds of pages. This is not about making sure a vanilla Plone setup works. This is about supporting large numbers of templates in all kinds of deployments that fully conform to the TAL/METAL specification but somehow fail under Chameleon 4.5 and up. Having to worry about an unknown number of templates potentially failing in unknown ways is a catastrophe. |
This was returning an ast.Expression() node. In Chameleon 4.4.4 that was handled correctly when generating code (https://github.com/malthe/chameleon/blob/4.4.4/src/chameleon/astutil.py#L295) In Chameleon >=4.5.0, it now uses ast.unparse for generating code. It does not explicitly handle Expression nodes, so it falls back to a generic visitor that clears the code generated so far. Then we got strange indentation errors while trying to run the template. Expression is meant to be a top-level node and shouldn't be used here. Returning the expression body also matches what the trusted PythonExpr does (https://github.com/malthe/chameleon/blob/master/src/chameleon/tales.py#L235)
|
I think I've resolved the problem with Chameleon 4.6.0 -- see my commit message on d5d3dd7 which I added here. But it looks like we need #1272 in order to make the tests run with zc.buildout 5. Btw, @mauritsvanrees, I think the reason you didn't see issues in Plone is because it was a bug in parsing untrusted Python expressions, and these days all templates in Plone core are trusted. |
|
@davisagli Thank you very much for finding and fixing the Chameleon issue. I would never have thought the fix is just one small change in Zope itself. |
|
@dataflake It took me about 4 hours of focused debugging to figure out where the problem was. Fortunately I had a flight yesterday, and I had fun. You're welcome! |
4 participants
We should use latest Chameleon (only possible on Python 3.9+) to avoid deprecation warnings for some changes in the
astmodule. In Python 3.14 these will become errors.See chameleon PR.
Zope itself also uses some deprecated
astcode inProducts.PageTemplates. Andz3c.ptdoes it as well. Maybe it needs to be fixed there first:At any rate the Zope tests currently fail with latest Chameleon:
I did not yet manage to get that generated python file and see what the source code is.