[CI] Remove redundant workflows (#4335)

### What this PR does / why we need it?
Remove redundant workflows， just maintain a separate workflow which
setting up on the main branch to control the execution of each branch,
instead of running each branch simultaneously, thus reducing resource
waste.


Signed-off-by: wangli <[email protected]>

Author: Potabk