Fix AV1 encoder availability for custom RTMP streaming services

[aleksandr-voitenko]

Description

Fix AV1 encoder availability for custom RTMP streaming services.

OBS-node previously treated a streaming service with no explicit supported video codec list as incompatible with all service-gated encoders. That caused AV1 encoders such as AOM AV1 and SVT-AV1 to be filtered out, or to fail start validation with "Invalid video encoder", when the runtime service was rtmp_custom.

This change mirrors OBS Studio’s behavior:

• keep explicit service codec lists authoritative
• when a service has no codec list, resolve the compatible streaming output from the service protocol/preferred output
• fall back to obs_get_output_supported_video_codecs(outputType) for codec compatibility

This keeps platform restrictions intact while allowing custom RTMP services to use codecs supported by their output, including AV1 over RTMP/RTMPS.

Motivation and Context

YouTube protected mode can use a custom RTMP service at runtime even though settings may query YouTube through rtmp_common. rtmp_custom does not expose obs_service_get_supported_video_codecs(), so OBS-node rejected AV1 encoders even though the RTMP output supports AV1.

OBS Studio already handles this case by falling back to output-supported codecs for custom/no-codec-list services.

OBS References

• obsproject/obs-studio#8088 - RFC 45-3: Use protocol for video codec selections
• obsproject/obs-studio#6802 - UI: Omit stream codecs the service doesn’t support
• obsproject/obs-studio#8522 - Enable AV1, HEVC via RTMP to YouTube

How Has This Been Tested?

QA, Windows only.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)