sagemathgh-38841: Fix `transformation` parameter for LLL on matrices over QQ
    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->
There was an oversight when interpreting the result of LLL() on an
integer matrix which lead to a type error. It assumed a matrix would
always be returned but if `transformation=True` was passed it would
instead be a tuple. This is now fixed and a test has been added.

If a `transformation` parameter is added for BKZ in the future this will
need to be done for it as well.


### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [x] I have created tests covering the changes.
- [x] I have updated the documentation and checked the documentation
preview.
    
URL: sagemath#38841
Reported by: TheBlupper
Reviewer(s):

Author: Release Manager

src/sage/matrix/matrix_rational_dense.pyx

@@ -2955,13 +2955,19 @@ cdef class Matrix_rational_dense(Matrix_dense):
             [ 1/28 -1/40 -1/18]
             [ 1/28 -1/40  1/18]
             [    0 -3/40     0]
+            sage: L, U = A.LLL(transformation=True)
+            sage: U * A == L
+            True
 
             sage: A = random_matrix(QQ, 10, 10)
             sage: d = lcm(a.denom() for a in A.list())
             sage: A.LLL() == (A * d).change_ring(ZZ).LLL() / d
             True
         """
         A, d = self._clear_denom()
+        if kwargs.get('transformation', False):
+            L, U = A.LLL(*args, **kwargs)
+            return L / d, U
         return A.LLL(*args, **kwargs) / d
 
     def is_LLL_reduced(self, delta=None, eta=None):