使用syngec_transformer生成脚本输出报错 #40
Description
./generate_syngec_transformer.sh
Generating MuCGEC Test...
2025-05-29 17:44:20 | INFO | fairseq_cli.interactive | Namespace(all_gather_list_size=16384, bart_model_file_from_transformers=None, batch_size=32, batch_size_valid=32, beam=12, bf16=False, bpe=None, broadcast_buffers=False, bucket_cap_mb=25, buffer_size=10000, checkpoint_shard_count=1, checkpoint_suffix='', conll_file=['../../preprocess/chinese_mucgec_with_syntax_transformer/bin/test.conll.src-tgt.src'], conll_suffix=['conll'], constraints=None, cpu=False, criterion='cross_entropy', curriculum=0, data='../../preprocess/chinese_hsk+lang8_with_syntax_transformer/bin', data_buffer_size=10, dataset_impl=None, ddp_backend='c10d', decoding_format=None, device_id=0, disable_validation=False, distributed_backend='nccl', distributed_init_method=None, distributed_no_spawn=False, distributed_num_procs=1, distributed_port=-1, distributed_rank=0, distributed_world_size=1, distributed_wrapper='DDP', diverse_beam_groups=-1, diverse_beam_strength=0.5, diversity_rate=-1.0, dpd_file=['../../preprocess/chinese_mucgec_with_syntax_transformer/bin/test.dpd.src-tgt.src'], dpd_suffix=['dpd'], empty_cache_freq=0, eval_bleu=False, eval_bleu_args=None, eval_bleu_detok='space', eval_bleu_detok_args=None, eval_bleu_print_samples=False, eval_bleu_remove_bpe=None, eval_tokenized_bleu=False, fast_stat_sync=False, find_unused_parameters=False, fix_batches_to_gpus=False, fixed_validation_seed=None, force_anneal=None, fp16=True, fp16_init_scale=128, fp16_no_flatten_grads=False, fp16_scale_tolerance=0.0, fp16_scale_window=None, gen_subset='test', input='-', iter_decode_eos_penalty=0.0, iter_decode_force_max_iter=False, iter_decode_max_iter=10, iter_decode_with_beam=1, iter_decode_with_external_reranker=False, left_pad_source='True', left_pad_target='False', lenpen=1, lm_path=None, lm_weight=0.0, load_alignments=False, localsgd_frequency=3, log_format='tqdm', log_interval=100, lr_scheduler='fixed', lr_shrink=0.1, match_source_len=False, max_len_a=0, max_len_b=200, max_source_positions=512, max_target_positions=1024, max_tokens=None, max_tokens_valid=None, memory_efficient_bf16=False, memory_efficient_fp16=False, min_len=1, min_loss_scale=0.0001, model_overrides='{}', model_parallel_size=1, nbest=1, no_beamable_mm=False, no_early_stop=False, no_progress_bar=False, no_repeat_ngram_size=0, no_seed_provided=False, nprocs_per_node=1, num_batch_buckets=0, num_shards=1, num_workers=12, optimizer=None, output_file='../../model/syngec/results/mucgec.out.nbest', path='../../model/syngec/chinese_transformer_syngec.pt', pipeline_balance=None, pipeline_checkpoint='never', pipeline_chunks=0, pipeline_decoder_balance=None, pipeline_decoder_devices=None, pipeline_devices=None, pipeline_encoder_balance=None, pipeline_encoder_devices=None, pipeline_model_parallel=False, prefix_size=0, print_alignment=False, print_step=False, probs_file=['../../preprocess/chinese_mucgec_with_syntax_transformer/bin/test.probs.src-tgt.src'], probs_suffix=['probs'], profile=False, quantization_config_path=None, quiet=False, remove_bpe='@@ ', replace_unk=None, required_batch_size_multiple=8, required_seq_len_multiple=1, results_path=None, retain_dropout=False, retain_dropout_modules=None, retain_iter_history=False, sacrebleu=False, sampling=False, sampling_topk=-1, sampling_topp=-1.0, score_reference=False, scoring='bleu', seed=1, shard_id=0, skip_invalid_size_inputs_valid_test=False, slowmo_algorithm='LocalSGD', slowmo_momentum=None, source_lang='src', source_lang_with_nt='src_nt', src_nt_file=None, swm_suffix='swm', syntax_encoder='GCN', syntax_model_file=None, syntax_type=['dep'], target_lang='tgt', task='syntax-enhanced-translation', temperature=1.0, tensorboard_logdir=None, threshold_loss_scale=None, tokenizer=None, tpu=False, train_subset='train', truncate_source=False, unkpen=0, unnormalized=False, upsample_primary=1, use_dpd=False, use_syntax=False, user_dir='../../src/src_syngec/syngec_model', valid_subset='valid', validate_after_updates=0, validate_interval=1, validate_interval_updates=0, warmup_updates=0, zero_sharding='none')
2025-05-29 17:44:20 | INFO | syngec_model.syntax_guided_gec_task | [src] dictionary: 21132 types
2025-05-29 17:44:20 | INFO | syngec_model.syntax_guided_gec_task | [tgt] dictionary: 21132 types
2025-05-29 17:44:20 | INFO | syngec_model.syntax_guided_gec_task | [syntax label0] dictionary: 51 types
2025-05-29 17:44:20 | INFO | fairseq_cli.interactive | loading model(s) from ../../model/syngec/chinese_transformer_syngec.pt
/home/zhangyiyin/下载/ENTER/envs/syngec1/lib/python3.8/site-packages/torch/cuda/init.py:146: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
2025-05-29 17:44:35 | INFO | fairseq_cli.interactive | Sentence buffer size: 10000
2025-05-29 17:44:35 | INFO | fairseq_cli.interactive | NOTE: hypothesis and token scores are output in base 2
2025-05-29 17:44:35 | INFO | fairseq_cli.interactive | Type the input sentence and press return:
0it [00:00, ?it/s]2025-05-29 17:44:35 | INFO | fairseq.data.data_utils | loaded 6739 examples from: ../../preprocess/chinese_mucgec_with_syntax_transformer/bin/test.conll.src-tgt.src
2025-05-29 17:44:35 | INFO | fairseq.data.data_utils | loaded 6739 examples from: ../../preprocess/chinese_mucgec_with_syntax_transformer/bin/test.dpd.src-tgt.src
2025-05-29 17:44:35 | INFO | fairseq.data.data_utils | loaded 6739 examples from: ../../preprocess/chinese_mucgec_with_syntax_transformer/bin/test.probs.src-tgt.src
2025-05-29 17:44:36 | INFO | fairseq.data.language_pair_dataset | success! syntax types: 1, source conll lines: 6739
0it [00:01, ?it/s]
Traceback (most recent call last):
File "../../src/src_syngec/fairseq-0.10.2/fairseq_cli/interactive.py", line 456, in
cli_main()
File "../../src/src_syngec/fairseq-0.10.2/fairseq_cli/interactive.py", line 452, in cli_main
distributed_utils.call_main(args, main)
File "/media/ExtHDD02/zhangyiyin/kinetics-dataset-main/SynGEC/src/src_syngec/fairseq-0.10.2/fairseq/distributed_utils.py", line 301, in call_main
main(args, **kwargs)
File "../../src/src_syngec/fairseq-0.10.2/fairseq_cli/interactive.py", line 315, in main
for batch in tqdm(make_batches(inputs, args, task, max_positions, encode_fn)):
File "/home/zhangyiyin/下载/ENTER/envs/syngec1/lib/python3.8/site-packages/tqdm/std.py", line 1181, in iter
for obj in iterable:
File "../../src/src_syngec/fairseq-0.10.2/fairseq_cli/interactive.py", line 156, in make_batches
for batch in itr:
File "/media/ExtHDD02/zhangyiyin/kinetics-dataset-main/SynGEC/src/src_syngec/fairseq-0.10.2/fairseq/data/iterators.py", line 59, in iter
for x in self.iterable:
File "/home/zhangyiyin/下载/ENTER/envs/syngec1/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 652, in next
data = self._next_data()
File "/home/zhangyiyin/下载/ENTER/envs/syngec1/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 692, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/zhangyiyin/下载/ENTER/envs/syngec1/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/zhangyiyin/下载/ENTER/envs/syngec1/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/media/ExtHDD02/zhangyiyin/kinetics-dataset-main/SynGEC/src/src_syngec/fairseq-0.10.2/fairseq/data/language_pair_dataset.py", line 776, in getitem
src_incoming_arc_mask_now = self.src_conll_dataset[i][index].view(l, l)
RuntimeError: shape '[5, 5]' is invalid for input of size 144
Generating Finish!