NullPointer EmbeddedMedia even on v1.1.0

[andehr]

Hi,
I'm getting the follow error:

java.lang.NullPointerException: Null type
	at net.dean.jraw.models.$AutoValue_EmbeddedMedia_OEmbed.<init>($AutoValue_EmbeddedMedia_OEmbed.java:45)
	at net.dean.jraw.models.AutoValue_EmbeddedMedia_OEmbed.<init>(AutoValue_EmbeddedMedia_OEmbed.java:16)
	at net.dean.jraw.models.AutoValue_EmbeddedMedia_OEmbed$MoshiJsonAdapter.fromJson(AutoValue_EmbeddedMedia_OEmbed.java:142)
	at net.dean.jraw.models.AutoValue_EmbeddedMedia_OEmbed$MoshiJsonAdapter.fromJson(AutoValue_EmbeddedMedia_OEmbed.java:19)

Which looks like it was resolved in version 1.1.0 (which I'm using via maven central) according to this issue: #227

Any idea how to fix or why this could be happening?

I can even see the code that patched it in my IDE:

@AutoValue
public abstract class EmbeddedMedia implements Serializable {
    /**
     * This is generally the OEmbed provider. Can also be the string {@code liveupdate} for a live reddit thread. Null
     * when {@link #getRedditVideo() is not}
     */
    @Nullable
    public abstract String getType();

Could AutoValue not be pulled in correctly?

In the autogenerated class $AutoValue_EmbeddedMedia_OEmbed I see this:

$AutoValue_EmbeddedMedia_OEmbed(String type, String version, @Nullable String title, @Nullable String authorName, @Nullable String authorUrl, @Nullable String providerName, @Nullable String providerUrl, @Nullable Long cacheAge, @Nullable String thumbnailUrl, @Nullable Integer thumbnailWidth, @Nullable Integer thumbnailHeight, @Nullable String url, @Nullable Integer width, @Nullable Integer height, @Nullable String embedHtml) {
        if (type == null) {
            throw new NullPointerException("Null type");
        }

This happens in a call to subreddits(...).posts(), after collecting about 44000 "NEW" posts from across ~80 subreddits. If I skip that page by catching the NullPointer, all subsequent calls to ".next()" result in the same error.

[andehr]

Is this because it's not the EmbeddedMedia type that is null, but actually the OEmbed type that is null? Which isn't marked as Nullable in the source:

/**
     * An object that models the JSON response for the oEmbed standard. Properties are made nullable if the spec deems
     * them optional. See [here](https://oembed.com/) for more.
     */
    @AutoValue
    public static abstract class OEmbed implements Serializable {
        /**
         * One of "photo", "video", "link", or "rich". See
         * [section 2.3.4.1 to 2.3.4.4 of the OEmbed standard](https://oembed.com) for more.
         */
        public abstract String getType();

[andehr]

Is the only necessary fix to mark that field as Nullable?

[andehr]

Here's the workaround that I'm using to avoid the need to change JRAW. I pass in a custom HTTP client, which intercepts requests and removes the offending JSON (oembed elements with null type fields) before JRAW sees it. I make use of JsonPath to do JSON manipulation, but you can use what you like.

// Custom client with interceptor which removes offending JSON before JRAW sees it
OkHttpClient httpClient = new OkHttpClient.Builder()
    .addInterceptor(new MalformedJsonInterceptor())
    .build();

// Build RedditClient
OkHttpNetworkAdapter adapter = new OkHttpNetworkAdapter(getUserAgent(), httpClient);
RedditClient reddit = OAuthHelper.automatic(adapter, credentials);

class MalformedJsonInterceptor implements Interceptor {

    @Override
    public Response intercept(Chain chain) throws IOException {
        Response response = chain.proceed(chain.request());
        return updateResponse(response, cleanJson(response.body().string()));
    }

    static final String oembedElements = "$..oembed[?]";
    static final Filter whereTypeIsNull = filter(where("type").exists(false));

    String cleanJson(String jsonString){
        return JsonPath.parse(jsonString)
            .delete(oembedElements, whereTypeIsNull)
            .jsonString();
    }

    Response updateResponse(Response previous, String newContent){
         MediaType contentType = previous.body().contentType();
         ResponseBody body = ResponseBody.create(contentType, newContent);
         return previous.newBuilder().body(body).build();
    }
}