0.20.0-RC4

Version 0.20.0-RC4 continues bug fixes and completing work related to "reactive pull" backpressure. This release updates amb and concat to connect the backpressure request.

Internal uses of RxRingBuffer migrated to using SpmcArrayQueue which significantly reduces object allocations. See #1526 for details.

The multicast operators were updated to use a Subject factory so that Observable sequences can be reused. See #1515 for details.

• Pull 1534 Concat Backpressure
• Pull 1533 Amb + Backpressure
• Pull 1527 Failing unit test for reduce, showing it does not implement backpressure correctly
• Pull 1528 Add operators to create Observables from BroadcastReceiver (rebased)
• Pull 1523 Fix issue #1522: takeLast
• Pull 1530 Fix the unbounded check for merge
• Pull 1526 Restore use of SpmcArrayQueue in RxRingBuffer
• Pull 1468 RxScala: Update CompletenessTest.scala
• Pull 1515 Support Subject Factory with Multicast
• Pull 1518 Fix typos in javadoc comments
• Pull 1521 Fix toIterator Exception Handling
• Pull 1520 Fix non-deterministic RxRingBuffer test

Artifacts: Maven Central

0.20.0-RC3

Version 0.20.0-RC3 preview release fixes several bugs related to backpressure and adds retryWhen, repeatWhen for more advanced recursion use cases like retry with exponential backoff.

This version passed the Netflix API production canary process. Please test this against your code to help us find any issues before we release 0.20.0.

• Pull 1493 retryWhen/repeatWhen
• Pull 1494 zipWith
• Pull 1501 blocking synchronous next
• Pull 1498 non-deterministic testUserSubscriberUsingRequestAsync
• Pull 1497 spsc ring buffer concurrency test
• Pull 1496 Change RxRingBuffer Queue Usage
• Pull 1491 Concat Outer Backpressure
• Pull 1490 non-deterministic timeouts on slow machines
• Pull 1489 Backpressure Fixes and Docs
• Pull 1474 Ignore backpressure for OperatorToObservableSortedList
• Pull 1473 OperatorAny needs to handle backpressure
• Pull 1472 Add test of backpressure to OperatorAll
• Pull 1469 ToList operator needs to ignore backpressure
• Pull 1393 Add cache(int capacity) to Observable
• Pull 1431 CompositeException fix for Android
• Pull 1436 Correct warnings

Artifacts: Maven Central

0.20.0-RC2

Version 0.20.0-RC2 preview release adds support for backpressure to the zip operators, fixes bugs and removes the Subscribe.onSetProducer method.

This means signature changes are modified to be:

The new type Producer ->

public interface Producer {
    public void request(long n);
}

New methods added to Subscriber ->

public abstract class Subscriber<T> implements Observer<T>, Subscription {
    public void onStart();
    protected final void request(long n);
    public final void setProducer(Producer producer);
}

• Pull 1448 RxScala: Add Scala idiomatic methods
• Pull 1446 Zip with Backpressure Support
• Pull 1454 doOnEachObserver fix
• Pull 1457 MergeDelayError & OnErrorFlatMap w/ Merge
• Pull 1458 Remove Pivot Operator
• Pull 1459 Remove Subscriber.onSetProducer
• Pull 1462 Merge Perf Fix: Re-enable fast-path
• Pull 1463 Merge Bug: Missing Emissions

Artifacts: Maven Central

0.20.0-RC1

Version 0.20.0-RC1 is a preview release that adds backpressure support to RxJava as per issue #1000. It has been done in a way that is mostly additive and most existing code will not be affected by these additions. A section below on "Breaking Changes" will discuss use cases that do break and how to deal with them.

This release has been tested successfully in Netflix production canaries, but that does not exercise all use cases or operators, nor does it leverage the newly added backpressure functionality (though the backpressure code paths are used).

Outstanding Work

• The zip operator has not yet been upgraded to support backpressure. The work is almost done and it will be included in the next release.
• Not all operators have yet been reviewed for whether they need to be changed in any way.
• Temporal operators (like buffer, window, sample, etc) need to be modified to disable backpressure upstream (using request(Long.MAX_VALUE)) and a decision made about how downstream backpressure requests will be supported.
• Ensure all code works on Android. New data structures rely on sun.misc.Unsafe but are conditionally used only when it is available. We need to ensure those conditions are working and the alternative implementations are adequate. The default buffer size of 1024 also needs to be reviewed for whether it is a correct default for all systems, or needs to be modified by environment (such as smaller for Android).
• Ensure use cases needing backpressure all work.

Signature Changes

A new type Producer has been added:

public interface Producer {
    public void request(long n);
}

The Subscriber type now has these methods added:

public abstract class Subscriber<T> implements Observer<T>, Subscription {
    public void onStart();
    public final void request(long n);
    public final void setProducer(Producer producer);
    protected Producer onSetProducer(Producer producer);
}

Examples

This trivial example shows requesting values one at a time:

Observable.from(1, 2, 3, 4).subscribe(new Subscriber<Integer>() {

    @Override
    public void onStart() {
        request(1);
    }

    @Override
    public void onCompleted() {
    }

    @Override
    public void onError(Throwable e) {
    }

    @Override
    public void onNext(Integer t) {
        request(1);
    }

});

The OnSubscribeFromIterable operator shows how an Iterable is consumed with backpressure.

Some hi-lights (modified for simplicity rather than performance and completeness):

public final class OnSubscribeFromIterable<T> implements OnSubscribe<T> {

    @Override
    public void call(final Subscriber<? super T> o) {
        final Iterator<? extends T> it = is.iterator();
        // instead of emitting directly to the Subscriber, it emits a Producer
        o.setProducer(new IterableProducer<T>(o, it));
    }

    private static final class IterableProducer<T> implements Producer {

        public void request(long n) {
            int _c = requested.getAndAdd(n);
            if (_c == 0) {
                while (it.hasNext()) {
                    if (o.isUnsubscribed()) {
                        return;
                    }
                    T t = it.next();
                    o.onNext(t);
                    if (requested.decrementAndGet() == 0) {
                        // we're done emitting the number requested so return
                        return;
                    }
                }

                o.onCompleted();
            }

        }
    }
}

The observeOn operator is a sterotypical example of queuing on one side of a thread and draining on the other, now with backpressure.

private static final class ObserveOnSubscriber<T> extends Subscriber<T> {
        @Override
        public void onStart() {
            // signal that this is an async operator capable of receiving this many
            request(RxRingBuffer.SIZE);
        }

        @Override
        public void onNext(final T t) {
            try {
                // enqueue
                queue.onNext(t);
            } catch (MissingBackpressureException e) {
                // fail if the upstream has not obeyed our backpressure requests
                onError(e);
                return;
            }
            // attempt to schedule draining if needed
            schedule();
        }

        // the scheduling polling will then drain the queue and invoke `request(n)` to request more after draining
}

Breaking Changes

The use of Producer has been added in such a way that it is optional and additive, but some operators that used to have unbounded queues are now bounded. This means that if a source Observable emits faster than the Observer can consume them, a MissingBackpressureException can be emitted via onError.

This semantic change can break existing code.

There are two ways of resolving this:

1. Modify the source Observable to use Producer and support backpressure.
2. Use newly added operators such as onBackpressureBuffer or onBackpressureDrop to choose a strategy for the source Observable of how to behave when it emits more data than the consuming Observer is capable of handling. Use of onBackpressureBuffer effectively returns it to having an unbounded buffer and behaving like version 0.19 or earlier.

Example:

sourceObservable.onBackpressureBuffer().subscribe(slowConsumer);

Relation to Reactive Streams

Contributors to RxJava are involved in defining the Reactive Streams spec. RxJava 1.0 is trying to comply with the semantic rules but is not attempting to comply with the type signatures. It will however have a separate module that acts as a bridge between the RxJava Observable and the Reactive Stream types.

The reasons for this are:

• Rx has Observer.onCompleted whereas Reactive Streams has onComplete. This is a massive breaking change to remove a "d".
• The RxJava Subscription is used also a "Closeable"/"Disposable" and it does not work well to make it now also be used for request(n), hence the separate type Producer in RxJava. It was attempted to reuse rx.Subscription but it couldn't be done without massive breaking changes.
• Reactive Streams uses onSubscribe(Subscription s) whereas RxJava injects the Subscription as the Subscriber. Again, this change could not be done without major breaking changes.
• RxJava 1.0 needs to be backwards compatible with the major Rx contracts established during the 0.x roadmap.
• Reactive Streams is not yet 1.0 and despite significant progress, it is a moving target.

Considering these things, the major semantics of request(long n) for backpressure are compatible and this will allow interop with a bridge between the interfaces. As the Reactive Streams spec matures, RxJava 2.0 may choose to fully adopt the types in the future while RxJava 1.x retains the current signatures.

How to Help

First, please test this release against your existing code to help us determine if we have broken anything.

Second, try to solve backpressure use cases and provide feedback on what works and what doesn't work.

Thank you!

Artifacts: Maven Central

0.19.6

Inclusion of rxjava-scalaz in release.

Artifacts: Maven Central

0.19.4

• Pull 1401 OnError while emitting onNext value: object.toString
• Pull 1409 Avoiding OperatorObserveOn from calling subscriber.onNext(..) after unsubscribe
• Pull 1406 Kotlin M8
• Pull 1400 Internal Data Structures
• Pull 1399 Update Perf Tests
• Pull 1396 RxScala: Fix the compiler warnings
• Pull 1397 Adding the hooks unsafeSubscribe

Artifacts: Maven Central

0.19.2

• Pull 1388 CompositeException stops mutating nested Exceptions
• Pull 1387 Upgrade to JMH 0.9
• Pull 1297 [RxScala] rxjava-scalaz: providing some type class instances
• Pull 1332 IOSSchedulers for RoboVM
• Pull 1380 Variety of Fixes
• Pull 1379 Parallel Operator Rewrite
• Pull 1378 BugFix: Pivot Concurrency
• Pull 1376 Revision of JMH Tests
• Pull 1375 RxScala: Add idiomatic toXXX methods
• Pull 1367 Fix the bug that 'flatMap' swallows OnErrorNotImplementedException
• Pull 1374 Fix head/tail false sharing issues
• Pull 1369 DebugHook got miswired before
• Pull 1361 Fix a race condition if queued actions have been handled already
• Pull 1336 RxScala: Add the rest missing methods to BlockingObservable
• Pull 1362 RxScala: Fix #1340 and #1343
• Pull 1359 Fixed padding of the integer and node classes

Artifacts: Maven Central

0.19.1

• Pull 1357 MergeWith, ConcatWith, AmbWith
• Pull 1345 RxScala: Simplify doOnCompleted/Terminate, finallyDo callback usage
• Pull 1337 Make Future receive NoSuchElementException when the BlockingObservable is empty
• Pull 1335 RxAndroid: Bump build tools to 19.1 and android plugin to 0.11
• Pull 1327 Join patterns extension for 4..9 and N arity joins.
• Pull 1321 RxAndroid: Ensuring Runnables posted with delay to a Handler are removed when unsubcribed
• Pull 1347 Allow use of the returned subscription to cancel periodic scheduling
• Pull 1355 Don't add the subscriber to the manager if it unsubscribed during the onStart call
• Pull 1350 Baseline Performance Tests
• Pull 1316 RxScala: Add the rest operators
• Pull 1324 TrampolineScheduler & Unsubscribe
• Pull 1311 Tiny integration test change

Artifacts: Maven Central

0.19.0

This release focused on performance and cleanup. If all goes according to plan the next release should be 0.20.0 with backpressure and then we will release 1.0 Release Candidate with the public API no longer allowing breaking changes.

Performance and Object Allocation

Fairly significant object allocation improvements are included in this release which reduce GC pressure and improve performance.

Two pull requests (amongst several) with details are:

• #1281 Reduce Subscription Object Allocation
• #1246 Moved to atomic field updaters

With the following simple test code relative performance has increased as shown below:

Observable<Integer> o = Observable.just(1);
o.map(i -> {
    return String.valueOf(i);
}).map(i -> {
    return Integer.parseInt(i);
}).subscribe(observer);

Rx 0.19

Run: 10 - 10,692,099 ops/sec 
Run: 11 - 10,617,627 ops/sec 
Run: 12 - 10,938,405 ops/sec 
Run: 13 - 10,917,388 ops/sec 
Run: 14 - 10,783,298 ops/sec 

Rx 0.18.4

Run: 11 - 8,493,506 ops/sec 
Run: 12 - 8,403,361 ops/sec 
Run: 13 - 8,400,537 ops/sec 
Run: 14 - 8,163,998 ops/sec 

Rx 0.17.6

Run: 10 - 4,930,966 ops/sec 
Run: 11 - 6,119,951 ops/sec 
Run: 12 - 7,062,146 ops/sec 
Run: 13 - 6,514,657 ops/sec 
Run: 14 - 6,369,426 ops/sec 

Rx 0.16.1

Run: 10 - 2,879,355 ops/sec 
Run: 11 - 3,236,245 ops/sec 
Run: 12 - 4,468,275 ops/sec 
Run: 13 - 3,237,293 ops/sec 
Run: 14 - 4,683,840 ops/sec 

Note that these numbers are relative as they depend on the JVM and hardware.

Scala Changes

Many missing operators have been added to the RxScala APIs along with fixes and other maturation.

toBlockingObservable() -> toBlocking()

The toBlockingObservable() method has been deprecated in favor of toBlocking() for brevity and fit better with possible future additions such as toParallel() without always needing the Observable suffix.

forEach

forEach as added as an alias for subscribe to match the Java 8 naming convention.

This means code can now be written as:

Observable.from(1, 2, 3).limit(2).forEach(System.out::println);

which is an alias of this:

Observable.from(1, 2, 3).take(2).subscribe(System.out::println);

Since forEach exists on BlockingObservable as well, moving from non-blocking to blocking looks like this:

// non-blocking
Observable.from(1, 2, 3).limit(2).forEach(System.out::println);
// blocking
Observable.from(1, 2, 3).limit(2).toBlocking().forEach(System.out::println);

Schedulers

Thread caching is restored to Schedulers.io() after being lost in v0.18.

A replacement for ExecutorScheduler (removed in 0.18) is accessible via Schedulers.from(Executor e) that wraps an Executor and complies with the Rx contract.

ReplaySubject

All "replay" functionality now exists directly on the ReplaySubject rather than in an internal type. This means there are now several different create methods with the various overloads of size and time.

Changelist

• Pull 1165 RxScala: Add dropUntil, contains, repeat, doOnTerminate, startWith, publish variants
• Pull 1183 NotificationLite.accept performance improvements
• Pull 1177 GroupByUntil to use BufferUntilSubscriber
• Pull 1182 Add facilities for creating Observables from JavaFX events and ObservableValues
• Pull 1188 RxScala Schedulers changes
• Pull 1175 Fixed synchronous ConnectableObservable.connect problem
• Pull 1172 ObserveOn: Change to batch dequeue
• Pull 1191 Fix attempt for OperatorPivotTest
• Pull 1195 SwingScheduler: allow negative schedule
• Pull 1178 Fix RxScala bug
• Pull 1210 Add more operators to RxScala
• Pull 1216 RxScala: Exposing PublishSubject
• Pull 1208 OperatorToObservableList: use LinkedList to buffer the sequence’s items
• Pull 1185 Behavior subject time gap fix 2
• Pull 1226 Fix bug in zipWithIndex and set zip(that, selector) public in RxScala
• Pull 1224 Implement shorter toBlocking as shorter alias for toBlockingObservable.
• Pull 1223 ReplaySubject enhancement with time and/or size bounds
• Pull 1160 Add replay and multicast variants to RxScala
• Pull 1229 Remove Ambiguous Subscribe Overloads with Scheduler
• Pull 1232 Adopt Limit and ForEach Java 8 Naming Conventions
• Pull 1233 Deprecate toBlockingObservable in favor of toBlocking
• Pull 1237 SafeSubscriber memory reduction
• Pull 1236 CompositeSubscription with atomic field updater
• Pull 1243 Remove Subscription Wrapper from Observable.subscribe
• Pull 1244 Observable.from(T) using Observable.just(T)
• Pull 1239 RxScala: Update docs for "apply" and add an example
• Pull 1248 Fixed testConcurrentOnNextFailsValidation
• Pull 1246 Moved to atomic field updaters.
• Pull 1254 ZipIterable unsubscription fix
• Pull 1247 Add zip(iterable, selector) to RxScala
• Pull 1260 Fix the bug that BlockingObservable.singleOrDefault doesn't call unsubscribe
• Pull 1269 Fix the bug that int overflow can bypass the range check
• Pull 1272 ExecutorScheduler to wrap an Executor
• Pull 1264 ObserveOn scheduled unsubscription
• Pull 1271 Operator Retry with predicate
• Pull 1265 Add more operators to RxScala
• Pull 1281 Reduce Subscription Object Allocation
• Pull 1284 Lock-free, MPSC-queue
• Pull 1288 Ensure StringObservable.from() does not perform unnecessary read
• Pull 1286 Rename some Operator* classes to OnSubscribe*
• Pull 1276 CachedThreadScheduler
• Pull 1287 ReplaySubject remove replayState CHM and related SubjectObserver changes
• Pull 1289 Schedulers.from(Executor)
• Pull 1290 Upgrade to JMH 0.7.3
• Pull 1293 Fix and Update JMH Perf Tests
• Pull 1291 Check unsubscribe within observable from future
• Pull 1294 rx.operators -> rx.internal.operators
• Pull 1295 Change void accept to boolean accept
• Pull 1296 Move re-used internal Scheduler classes to their own package
• Pull 1298 Remove Bad Perf Test
• Pull 1301 RxScala: Add convenience method for adding unsubscription callback
• Pull 1304 Add flatMap and concatMap to RxScala
• Pull 1306 Hooked RxJavaPlugins errorHandler up within all operators that swallow onErrors
• Pull 1309 Hide ChainedSubscription/SubscriptionList from Public API

Artifacts: Maven Central

0.18.4

This is a fix for CompositeSubscription object allocation problems. Details can be found in issue #1204.

• Pull 1283 Subscription object allocation fix

Artifacts: Maven Central