2.8.12

Added:

luigi

• EnumListParameter #2801

Fixed:

luigi

• Import ABC from collections.abc instead of collections for Python 3.9 compatibility #2895

Changed:

luigi.contrib

• [luigi.contrib.hive] HiveTableTarget inherits HivePartitionTarget #2872
• [luigi.contrib.pyspark_runner] SparkSession support in PySparkTask #2862

3.0.0b2

This the second 3.0.0 beta release including a series of PRs deprecating Python2, plus following:

Special thanks go to @drowoseque for doing all the great work!

Added:

luigi

• Add internal version info #2760
• EnumListParameter #2801 (new since 3.0.0b1)

luigi.contrib

• [luigi.contrib.spark] pyspark python options added #2818

Fixed:

luigi

• Fix params hashing #2540
• Check for autoload_range istead of autoload-range
• autoload_range doc fix #2878 (new since 3.0.0b1)

Removed:

luigi

• [luigi.file] removed #2832
• [luigi.mock.MockFile] removed #2839

Changed:

luigi

• Allow python-daemon >= 2.2.0 if not on windows #2796
• Make URLLibFetcher aware of basic auth info in scheduler URL. #2791

luigi.contrib

• [luigi.contrib.external_program.ExternalProgramTask] logs_output_pattern_to_url provided #2822
• [luigi.contrib.hive] HiveTableTarget inherits HivePartitionTarget #2872 (new since 3.0.0b1)
• [luigi.contrib.pyspark_runner] SparkSession support in PySparkTask #2862 (new since 3.0.0b1)

2.8.11

Added:

luigi

• Add internal version info #2760

luigi.contrib

• [luigi.contrib.spark] pyspark python options added #2818

Fixed:

luigi

• Fix params hashing #2540
• Check for autoload_range istead of autoload-range
• autoload_range doc fix #2878

Removed:

luigi

• [luigi.file] removed #2832
• [luigi.mock.MockFile] removed #2839

Changed:

luigi

• Allow python-daemon >= 2.2.0 if not on windows #2796
• Make URLLibFetcher aware of basic auth info in scheduler URL. #2791

luigi.contrib

• [luigi.contrib.external_program.ExternalProgramTask] logs_output_pattern_to_url provided #2822

2.8.10

Added:

luigi

• Add HEAD endpoint to scheduler server for status/health checks #2789

luigi.contrib

• [luigi.contrib.hive] WarehouseHiveClient #2826

Fixed:

luigi.contrib

• Add Python version-agnostic get_writer_schema. #2827
• PySparkTask: handle special characters in name (#2778) #2779

Changed:

luigi.contrib

• [luigi.contrib.spark] tracking_url_pattern as a property #2820
• Add pod_creation_wait_interal #2813
• Added optional argument 'aws_session_token' to S3Client #2798

2.8.9

Added:

luigi

• Adds "Force Commit" button in UI to set tasks to DONE #2751
• Show task history link in visualizer when recording. #2759

Fixed:

luigi

• Replace documentation reference to outdated test environment py27-nonhdfs #2762
• Issue 2644: Tasks can be run several times under certain conditions #2645

luigi & luigi.contrib

• Ensure ignored tests are picked up by tox #2758

Changed:

luigi

• Update tornado requirement for new enough python versions #2761

luigi.contrib

• contrib/ftp: Clean up temporary files #2755

2.8.8

Added:

luigi

• Expandable Namespace Folders for the Visualiser Sidebar #2716
• Added new companies to the luigi users list: #2730 #2747

luigi.contrib

• Enable Overriding Poll Interval for Kubernetes Jobs #2724

Fixed:

luigi

• Code example correction #2754

Changed:

luigi

• Update release process #2727
• Change GET request to POST requests in luigi/rpc #2732
• Fix SendGrid email API documentation. #2745

2.8.7

Added:

luigi

• Add check_complete_on_run to optionally mark tasks as failed if complete() is false when run finishes (#2710)
• Add section "Running Luigi on Windows" to docs (#2720)
• Add Giphy to list of companies using Luigi to docs (#2713)

luigi.contrib

• Add DropboxTarget for luigi (#2696)

Fixed:

luigi

• UI: Fix time graph - y axis to account for timezones. (#2711)

Changed:

luigi

• Bump dependencies used by SendGrid integration. (#2715)
• Update copyright year in LICENSE (#2723)

luigi.contrib

• Make RedisTarget compatible with redis-py >= 3 (#2722)

2.8.6

Fixed:

luigi

• fix calling method on wrong object #2709
  This fixes #2704.

2.8.5

Fixed:

luigi

• (Doc) Fix example of "summary_length" #2700
• Fix __init__ error when using TOML config #2702
• add callback to metric collector #2704

luigi.contrib

• Fix BigQueryTarget parsing in beam_dataflow module #2705

Changed:

luigi.contrib

• aws batch : job queue as parameter #2689

2.8.4 (BROKEN)

This release is broken due to #2628 .

Added:

luigi

• Added support for a detailed LuigiRunResult instead of a plain Boolean (#2630)
• Add worker option 'max_keep_alive_idle_duration (#2654)
• Added worker-id commandline parameter (#2655)

luigi.contrib

• Add support for specifying kubernetes namespace (#2629)
• Add a Task wrapper for MicroSoft OpenPAI (#2531)
• Provide automatic URL tracking for Spark applications (#2661) (#2669)
• add beam_dataflow_task to luigi/contrib (#2675)

Both

• Add Prometheus contrib for monitoring purpose (#2628)

Fixed:

luigi

• setup.py: Support older setuptools (<=20.1.1) (#2623)
• Fix broken aws tests (#2658)
• Accept pathlib based path as argument for LocalTarget (#2548)
• Fix durations in D3 graph (fixes #2620) (#2624)
• Import collections ABCs from collections.abc, not collections (#2683)
• Configuration documentation: remove deprecated/wrong [core]max_reschedules entry (#2692)

luigi.contrib

• fixes #2223 HdfsTarget is not working with snakebite (#2572)
• Add port field for PostgresQuery (fixes #2625) (#2627)

Both

• Fix flake errors after moving to python 3 (#2695)

Changed:

luigi

• Simplify implementation of temporary_path() (#2652)
• Prevent range-tasks from autoloading (#2656)
• Replace Python MapReduce example with Spark example. (#2668)
• Minor improvements to single-worker-timeout support (#2667)
• Require at least python-dateutil version 2.7.5 instead of only 2.7.5 (fixes #2662) (#2679)
• RangeMonthly should deal with whole months (#2666)
• Reconcile underscore/dash config style handling (#2688) #2691

luigi.contrib

• Add autodetect parameter to BigQueryLoadTask (#2363) (#2575)