Fix up tests (#243)

* Fix up tests

1. add test runs for python 3.7, remove 3.4
2. fix 2.7 tests to not use celery versions that don't support it
3. add pytest-timeout to avoid hanging certain tests when broker is not running

* Fix python 3.7 for travis-ci, fix flake8 error, move more things to 3.7

* yml formatting

* Put the 3.7 options in the right location of the travis-ci.yml

* celery python2 limit is 5.0 so properly set that.

* Attempt to bend travis-ci to my will with python 3.7

* load python2 requriements for pypy

* Properly match python version between tox.ini and travis.yml for doc/analysis tasks.

* Use older version of django for doc/cov due to sqlite incompatibility

* Fix a flake8 error, and fix django dependency for coverage

* Try setting hostname on containers to fix rabbitmq-server on xenial

* debugging rabbitmq on xenial in travis container

* More python37 in travis-ci debugging

* More python37 debugging

* python37 try updating erlang to make rabbitmq work

* Fix installation commands for erlang install on xenial

* python37 also manually install celery from latest master, allow external commands

* More python37 hackup fixing

Author: liquidpele

.travis.yml

@@ -3,25 +3,32 @@ sudo: false
 cache: false
 python:
   - 2.7
-  - 3.4
   - 3.5
   - pypy
 env:
   - DJANGO=1.11
   - DJANGO=2.0
+  - DJANGO=2.2
 os:
   - linux
 matrix:
   include:
-    - { python: 2.7, env: TOXENV=flake8 }
+    - {python: 3.7, env: DJANGO=1.11 TOXENV=python37, dist: xenial, sudo: true}
+    - {python: 3.7, env: DJANGO=2.0 TOXENV=python37, dist: xenial, sudo: true}
+    - {python: 3.7, env: DJANGO=2.2 TOXENV=python37, dist: xenial, sudo: true}
+    - { python: 3.5, env: TOXENV=flake8 }
     - { python: 2.7, env: TOXENV=flakeplus }
-    - { python: 2.7, env: TOXENV=pydocstyle }
-    - { python: 3.5, env: TOXENV=apicheck }
-    - { python: 3.5, env: TOXENV=linkcheck }
+    - { python: 3.5, env: TOXENV=pydocstyle }
     - { python: 3.5, env: TOXENV=cov }
+    # disabled temporarily due to upstream bug
+    # https://github.com/celery/sphinx_celery/issues/9
+    # - { python: 3.5, env: TOXENV=apicheck }
+    # - { python: 3.5, env: TOXENV=linkcheck }
   exclude:
     - { python: 2.7, env: DJANGO=2.0 }
+    - { python: 2.7, env: DJANGO=2.2 }
     - { python: pypy, env: DJANGO=2.0 }
+    - { python: pypy, env: DJANGO=2.2 }
 install: travis_retry pip install -U tox-travis
 services: rabbitmq
 script: tox -v -- -v

django_celery_beat/schedulers.py

@@ -295,7 +295,7 @@ def sync(self):
                 try:
                     self.schedule[name].save()
                     _tried.add(name)
-                except (KeyError, ObjectDoesNotExist) as exc:
+                except (KeyError, ObjectDoesNotExist):
                     _failed.add(name)
         except DatabaseError as exc:
             logger.exception('Database error while sync: %r', exc)

requirements/default.txt

@@ -1,3 +1,2 @@
-celery>=4.0,<5.0
 django-timezone-field>=2.0
 python-crontab>=2.3.4

requirements/docs.txt

@@ -1,2 +1,4 @@
+Django==1.11
 https://github.com/celery/sphinx_celery/archive/master.zip
-Django>=1.10
+https://github.com/celery/kombu/zipball/master#egg=kombu
+https://github.com/celery/celery/zipball/master#egg=celery

requirements/pkgutils.txt

@@ -5,4 +5,4 @@ flakeplus>=1.1
 tox>=2.3.1
 sphinx2rst>=1.0
 bumpversion
-pydocstyle
+pydocstyle

requirements/python2.txt

@@ -0,0 +1,3 @@
+# celery stops supporting python2 at 5.0.0
+# thus for python2 tests we can't use latest master anymore
+celery<5.0.0

requirements/python3.txt

@@ -0,0 +1,3 @@
+# for python3, install latest stuff
+https://github.com/celery/celery/zipball/master#egg=celery
+https://github.com/celery/kombu/zipball/master#egg=kombu

requirements/test-django20.txt

@@ -1 +1 @@
-django>=2.0,<2.1
+django>=2.0,<2.1

requirements/test.txt

@@ -1,4 +1,5 @@
 case>=1.3.1
-pytest>=3.0
 pytest-django
 pytz>dev
+pytest<4.0.0
+pytest-timeout

t/unit/test_schedulers.py

@@ -652,6 +652,9 @@ def patch_request(self, request):
         setattr(request, '_messages', messages)
         return request
 
+    # don't hang if broker is down
+    # https://github.com/celery/celery/issues/4627
+    @pytest.mark.timeout(5)
     def test_run_task(self):
         ma = PeriodicTaskAdmin(PeriodicTask, self.site)
         self.request = self.patch_request(self.request_factory.get('/'))
@@ -660,6 +663,9 @@ def test_run_task(self):
         queued_message = self.request._messages._queued_messages[0].message
         assert queued_message == '1 task was successfully run'
 
+    # don't hang if broker is down
+    # https://github.com/celery/celery/issues/4627
+    @pytest.mark.timeout(5)
     def test_run_tasks(self):
         ma = PeriodicTaskAdmin(PeriodicTask, self.site)
         self.request = self.patch_request(self.request_factory.get('/'))