[FEAT] Add `bitstream` module in `lib_ccxr` #1649

mrswastik-robot · 2024-12-25T13:02:05Z

In raising this pull request, I confirm the following (please check boxes):

I have read and understood the contributors guide.
I have checked that another pull request for this purpose does not exist.
I have considered, and confirmed that this submission will be valuable to others.
I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
I give this submission freely, and claim no ownership to its content.
I have mentioned this change in the changelog.

My familiarity with the project is as follows (check one):

I have never used CCExtractor.
I have used CCExtractor just a couple of times.
I absolutely love CCExtractor, but have not contributed previously.
I am an active contributor to CCExtractor.

This PR migrates the bitstream functionality from C to Rust while maintaining backward compatibility by creating a new Rust implementation in lib_ccxr/src/common/bitstream.rs and connecting it to the existing C code through FFI bindings.

prateekmedia · 2024-12-26T18:58:56Z

src/lib_ccx/cc_bitstream.h

 	unsigned char *_i_pos;
 	int _i_bpos;
+#ifndef DISABLE_RUST
+	void *rust_bs;  // Pointer to Rust's Bitstream struct


Could we construct / deconstruct the struct when passing to rust instead of storing a different pointer?

There are other files, where you can see the to_ctype()

yaa surely, this should be a better approach rather than having to maintain two structures. I will try to fix it in my next commit

@prateekmedia , I have made changes in the bitstream.rs file , can you please review it now

will check this.

Migration module changes

Minor Formatting Issue

Copilot

Pull Request Overview

This PR adds a new bitstream module implemented in Rust and connects it to the existing C code via FFI.

Introduces Rust-side bitstream functions and exposes them through the C wrapper
Updates C bitstream implementation to delegate to Rust when DISABLE_RUST is not defined
Adjusts tests and build configuration to include the new module

Reviewed Changes

Copilot reviewed 7 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/rust/wrapper.h	Includes the C header for bitstream to expose FFI
src/rust/src/libccxr_exports/mod.rs	Registers the `bitstream` export module
src/rust/lib_ccxr/src/util/bits.rs	Refactors parity tests to use `assert!`
src/rust/lib_ccxr/src/common/mod.rs	Adds `bitstream` to the common mod exports
src/rust/build.rs	Links the `bitstream` library into the build script
src/lib_ccx/cc_bitstream.c	Wraps the C implementation to call into Rust

Comments suppressed due to low confidence (3)

src/lib_ccx/cc_bitstream.c:238

The function signature returns unsigned char * but the Rust FFI ccxr_next_bytes exports a const uint8_t *. Consider updating this to const unsigned char * to preserve the const qualifier and avoid discarding it.

		bstr->bpos = 8;

src/rust/wrapper.h:14

wrapper.h includes the C header for bitstream but does not declare the extern ccxr_* functions provided by the Rust bitstream module, so consumers won’t see the Rust FFI symbols. Consider adding extern declarations for each ccxr_* function in this header.

#include "../lib_ccx/cc_bitstream.h"

src/rust/build.rs:30

The bitstream entry may not match the actual C library name produced (likely something like ccx_bitstream or part of the existing ccx library), which could cause linker errors. Verify that the linked library name matches the compiled archive.

        "bitstream",

src/rust/lib_ccxr/src/util/bits.rs

src/rust/lib_ccxr/src/common/mod.rs

src/rust/lib_ccxr/src/common/bitstream.rs

Bitstream: Removed redundant CType

src/rust/lib_ccxr/src/common/bitstream.rs

…-flow bitstream: recommended changes for is_byte_aligned

prateekmedia · 2025-07-06T13:54:36Z

src/rust/src/libccxr_exports/bitstream.rs

+/// # Safety
+/// This function is unsafe because it:
+/// - Dereferences a raw pointer (`bitstream_ptr`) without null checking
+/// - Assumes `bitstream_ptr` points to a valid, properly initialized and aligned `bitstream` struct
+/// - Assumes the `bitstream` struct has sufficient memory allocated for all fields
+/// - Assumes `rust_bitstream` is properly initialized with valid data
+/// - Performs pointer arithmetic on `rust_bitstream.data` assuming it points to valid memory
+/// - The caller must ensure the `bitstream` struct remains valid for the duration of the operation
+/// - The data referenced by `rust_bitstream.data` must remain valid during the operation


Keep Safety comments concise to 1 or 2 lines at max.

…-removal bitstream: recommended changes for long comments

prateekmedia

LGTM! Thanks.

* feat: Add bitstream module * run code formatters * Run cargo clippy --fix * Run cargo fmt --all * refactor: remove rust pointer from C struct * feat: Add bitstream module * run code formatters * Run cargo clippy --fix * Run cargo fmt --all * refactor: remove rust pointer from C struct * Added Bitstream to libccxr_exports * Minor Formatting Issue * Bitstream: Removed redundant CType * bitstream: recommended changes for is_byte_aligned * bitstream: recommended changes for long comments * bitstream: comment fix * bitstream: removed redundant comparism comments --------- Co-authored-by: Deepnarayan Sett <[email protected]> Co-authored-by: Deepnarayan Sett <[email protected]>

* feat: added demuxer module * Cargo Lock Update * Completed file_functions and demuxer * Completed file_functions and demuxer * written extern functions for demuxer * Removed libc completely, added tests for gxf and ported gxf to C * Hardsubx error fixed * Fixing format issues * clippy errors fixed * fixing format issues * fixing format issues * Windows failing tests * Windows failing tests * demuxer: added demuxer data transfer functions and removed some structs * made Demuxer and File Functions * Minor formatting changes * Minor Rebasing changes * demuxer: format rust and unit test rust checks * C formatting * Windows Failing test * Windows Failing test * Update CHANGES.TXT * Update CHANGES.TXT * Windows Failing Tests * Windows Failing Tests * Problem in Copy to Rust and some typos that copilot review suggested * Minor Formatting Error * Windows Failing Regressions * Windows Failing Regressions * Minor Comment Change * Data transfer module for DemuxerData added and more rustlike syntax to ctorust.rs * Minor Formatting Changes * demuxer: Rebase and a few tweaks to file_functions * demuxer: Minor Formatting Error * [FIX] 134 Codes in XDS and General Tests (#1708) * Made pointers valid in Unit Tests of Decoder * fix: test_do_cb * Copilot Suggestions * Suggestions about Redundancy * Suggestions about Redundancy * [FEAT] Add `bitstream` module in `lib_ccxr` (#1649) * feat: Add bitstream module * run code formatters * Run cargo clippy --fix * Run cargo fmt --all * refactor: remove rust pointer from C struct * feat: Add bitstream module * run code formatters * Run cargo clippy --fix * Run cargo fmt --all * refactor: remove rust pointer from C struct * Added Bitstream to libccxr_exports * Minor Formatting Issue * Bitstream: Removed redundant CType * bitstream: recommended changes for is_byte_aligned * bitstream: recommended changes for long comments * bitstream: comment fix * bitstream: removed redundant comparism comments --------- Co-authored-by: Deepnarayan Sett <[email protected]> Co-authored-by: Deepnarayan Sett <[email protected]> * demuxer: minor formatting changes * Demuxer: Changes to mistakes in CHANGES.txt * Demuxer: Removed extra newline in ccextractor.c * Demuxer: Changes to Encoding resolved * Demuxer: Moved CCX_NOPTS to common structs and some changes to Demuxer Data regd. MPEG_CLOCK_FREQ * some refactoring to CCX_NOPTS * Demuxer: Minor Mistake regarding CHANGES.txt * Demuxer: Unit test rust failing because of CCX_NOPTS * Demuxer: changed common_structs to common_types * Demuxer: Removed redundant libraries from Cargo.toml and moved tempfile to dev-dependencies * Demuxer: Removed to_vec function and renamed PSIBuffer/PMTEntry from_ctype functions * Demuxer: Renamed Stream_Type, improved Time complexity of the default() function and removed redundant comments * Demuxer: Removed two repeated code blocks and removed redundant comments * Demuxer: Removed two code blocks * Demuxer: Review Changes * Demuxer: Removed redundant tests * Update src/rust/src/demuxer/demux.rs Co-authored-by: Prateek Sunal <[email protected]> * Demuxer: Errors due to Rebase * Demuxer: Removed get_stream_mode * Demuxer: Errors due to rebasing and removing redundant CType Functions * Demuxer: Failing ES regressions * Demuxer: MythTV failing regression * Demuxer: Removed redundant comments * Demuxer: Unplugged ES for now * Demuxer: Replugged in ES * Demuxer: Formatting error * Demuxer: Windows failing CI * Demuxer: Windows failing CI * Demuxer: Windows failing Regressions * Demuxer: Formatting * Demuxer: Minor Cargo Clippy change * Demuxer: running regressions again * Demuxer: Cargo Lockfile Change * Demuxer: running regressions again * Demuxer: running regressions again --------- Co-authored-by: Swastik Patel <[email protected]> Co-authored-by: Prateek Sunal <[email protected]>

* feat: added demuxer module * Cargo Lock Update * Completed file_functions and demuxer * Completed file_functions and demuxer * written extern functions for demuxer * Removed libc completely, added tests for gxf and ported gxf to C * Hardsubx error fixed * Fixing format issues * clippy errors fixed * fixing format issues * fixing format issues * Windows failing tests * Windows failing tests * demuxer: added demuxer data transfer functions and removed some structs * made Demuxer and File Functions * Minor formatting changes * Minor Rebasing changes * demuxer: format rust and unit test rust checks * C formatting * Windows Failing test * Windows Failing test * Update CHANGES.TXT * Update CHANGES.TXT * Windows Failing Tests * Windows Failing Tests * Problem in Copy to Rust and some typos that copilot review suggested * Minor Formatting Error * Windows Failing Regressions * Windows Failing Regressions * Minor Comment Change * Data transfer module for DemuxerData added and more rustlike syntax to ctorust.rs * Minor Formatting Changes * demuxer: Rebase and a few tweaks to file_functions * demuxer: Minor Formatting Error * [FIX] 134 Codes in XDS and General Tests (CCExtractor#1708) * Made pointers valid in Unit Tests of Decoder * fix: test_do_cb * Copilot Suggestions * Suggestions about Redundancy * Suggestions about Redundancy * [FEAT] Add `bitstream` module in `lib_ccxr` (CCExtractor#1649) * feat: Add bitstream module * run code formatters * Run cargo clippy --fix * Run cargo fmt --all * refactor: remove rust pointer from C struct * feat: Add bitstream module * run code formatters * Run cargo clippy --fix * Run cargo fmt --all * refactor: remove rust pointer from C struct * Added Bitstream to libccxr_exports * Minor Formatting Issue * Bitstream: Removed redundant CType * bitstream: recommended changes for is_byte_aligned * bitstream: recommended changes for long comments * bitstream: comment fix * bitstream: removed redundant comparism comments --------- Co-authored-by: Deepnarayan Sett <[email protected]> Co-authored-by: Deepnarayan Sett <[email protected]> * demuxer: minor formatting changes * Demuxer: Changes to mistakes in CHANGES.txt * Demuxer: Removed extra newline in ccextractor.c * Demuxer: Changes to Encoding resolved * Demuxer: Moved CCX_NOPTS to common structs and some changes to Demuxer Data regd. MPEG_CLOCK_FREQ * some refactoring to CCX_NOPTS * Demuxer: Minor Mistake regarding CHANGES.txt * Demuxer: Unit test rust failing because of CCX_NOPTS * Demuxer: changed common_structs to common_types * Demuxer: Removed redundant libraries from Cargo.toml and moved tempfile to dev-dependencies * Demuxer: Removed to_vec function and renamed PSIBuffer/PMTEntry from_ctype functions * Demuxer: Renamed Stream_Type, improved Time complexity of the default() function and removed redundant comments * Demuxer: Removed two repeated code blocks and removed redundant comments * Demuxer: Removed two code blocks * Demuxer: Review Changes * Demuxer: Removed redundant tests * Update src/rust/src/demuxer/demux.rs Co-authored-by: Prateek Sunal <[email protected]> * Demuxer: Errors due to Rebase * Demuxer: Removed get_stream_mode * Demuxer: Errors due to rebasing and removing redundant CType Functions * Demuxer: Failing ES regressions * Demuxer: MythTV failing regression * Demuxer: Removed redundant comments * Demuxer: Unplugged ES for now * Demuxer: Replugged in ES * Demuxer: Formatting error * Demuxer: Windows failing CI * Demuxer: Windows failing CI * Demuxer: Windows failing Regressions * Demuxer: Formatting * Demuxer: Minor Cargo Clippy change * Demuxer: running regressions again * Demuxer: Cargo Lockfile Change * Demuxer: running regressions again * Demuxer: running regressions again --------- Co-authored-by: Swastik Patel <[email protected]> Co-authored-by: Prateek Sunal <[email protected]>

mrswastik-robot added 4 commits December 25, 2024 18:16

feat: Add bitstream module

83fb2d7

run code formatters

c3a91a6

Run cargo clippy --fix

c873193

Run cargo fmt --all

37e126a

prateekmedia reviewed Dec 26, 2024

View reviewed changes

mrswastik-robot and others added 11 commits December 27, 2024 21:53

refactor: remove rust pointer from C struct

666a1bf

feat: Add bitstream module

d70fcd3

run code formatters

e9a9155

Run cargo clippy --fix

bdb0c5f

Run cargo fmt --all

d869d7c

refactor: remove rust pointer from C struct

04a0eed

Added Bitstream to libccxr_exports

e04b430

Merge branch 'migration-bitstream-module' into migration-module-changes

0ebccad

Merge pull request #1 from steel-bucket/migration-module-changes

b1284d2

Migration module changes

Minor Formatting Issue

22a7259

Merge pull request #2 from steel-bucket/bitstream-fmt-issue

4ac2792

Minor Formatting Issue

prateekmedia requested a review from Copilot June 21, 2025 09:05

Copilot AI reviewed Jun 21, 2025

View reviewed changes

src/rust/lib_ccxr/src/util/bits.rs Show resolved Hide resolved

src/rust/lib_ccxr/src/common/mod.rs Show resolved Hide resolved

prateekmedia reviewed Jun 29, 2025

View reviewed changes

src/rust/lib_ccxr/src/common/bitstream.rs Outdated Show resolved Hide resolved

steel-bucket and others added 2 commits June 30, 2025 15:50

Bitstream: Removed redundant CType

d1b7984

Merge pull request #4 from steel-bucket/fix-bitstream-redundant-ctype

ca5aafd

Bitstream: Removed redundant CType

prateekmedia reviewed Jul 6, 2025

View reviewed changes

src/rust/lib_ccxr/src/common/bitstream.rs Outdated Show resolved Hide resolved

steel-bucket and others added 2 commits July 6, 2025 19:04

bitstream: recommended changes for is_byte_aligned

420979c

Merge pull request #5 from steel-bucket/bitstream-recommended-control…

90de368

…-flow bitstream: recommended changes for is_byte_aligned

prateekmedia reviewed Jul 6, 2025

View reviewed changes

steel-bucket and others added 4 commits July 6, 2025 20:51

bitstream: recommended changes for long comments

befe801

bitstream: comment fix

18d0b11

bitstream: removed redundant comparism comments

4fbb0ce

Merge pull request #6 from steel-bucket/bitstream-recommended-comment…

8a0a587

…-removal bitstream: recommended changes for long comments

prateekmedia approved these changes Jul 6, 2025

View reviewed changes

This comment was marked as off-topic.

Sign in to view

prateekmedia merged commit 81fdecd into CCExtractor:master Jul 6, 2025
18 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[FEAT] Add `bitstream` module in `lib_ccxr` #1649

[FEAT] Add `bitstream` module in `lib_ccxr` #1649

Uh oh!

mrswastik-robot commented Dec 25, 2024

Uh oh!

prateekmedia Dec 26, 2024

Uh oh!

mrswastik-robot Dec 26, 2024

Uh oh!

mrswastik-robot Dec 29, 2024

Uh oh!

prateekmedia Dec 29, 2024

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

prateekmedia Jul 6, 2025

Uh oh!

prateekmedia left a comment

Uh oh!

This comment was marked as off-topic.

This comment was marked as off-topic.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[FEAT] Add bitstream module in lib_ccxr #1649

[FEAT] Add bitstream module in lib_ccxr #1649

Uh oh!

Conversation

mrswastik-robot commented Dec 25, 2024

Uh oh!

prateekmedia Dec 26, 2024

Choose a reason for hiding this comment

Uh oh!

mrswastik-robot Dec 26, 2024

Choose a reason for hiding this comment

Uh oh!

mrswastik-robot Dec 29, 2024

Choose a reason for hiding this comment

Uh oh!

prateekmedia Dec 29, 2024

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

prateekmedia Jul 6, 2025

Choose a reason for hiding this comment

Uh oh!

prateekmedia left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

This comment was marked as off-topic.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[FEAT] Add `bitstream` module in `lib_ccxr` #1649

[FEAT] Add `bitstream` module in `lib_ccxr` #1649