Saved model .cfg file not matching the one used to create the model

[Vfgandara]

Hi there,
I'm training a NER model using a very simple pipeline, with just tok2vec and ner. I've been testing/trying to implement tok2vecListeners on the NER model and letting the model train both NER and tok2vec. Both of these aren't trained from the ground up, but rather are sourced from pt_core_news_lg. The models are performing well and all, but I'm confused about how it is behving, since the outputted config.cfg that is generated when the model finishes training doesn't seem to match the one I used to create it in the first place. The thing I don't truly understand is whether some of my configs are off an then the model's config gets overwritten because of that or if it's simply a bug.

Here's the config file I'm using:

[paths]
train = "train.spacy"
dev = "test.spacy"
vectors = null
init_tok2vec = null

[system]
gpu_allocator = null
seed = 0

[nlp]
lang = "pt"
pipeline = ["tok2vec","ner"]
batch_size = 1000
disabled = []
before_creation = null
after_creation = null
after_pipeline_creation = null
tokenizer = {"@tokenizers":"spacy.Tokenizer.v1"}

[components]

[components.ner]
source = "pt_core_news_lg"
incorrect_spans_key = null
moves = null
update_with_oracle_cut_size = 100

[components.ner.model]
@architectures = "spacy.TransitionBasedParser.v2"
state_type = "ner"
extra_state_tokens = false
hidden_width = 64
maxout_pieces = 2
use_upper = true
nO = null

[components.ner.model.tok2vec]
@architectures = "spacy.Tok2VecListener.v2"

[components.ner.model.tok2vec.embed]
@architectures = "spacy.MultiHashEmbed.v2"
width = 96
attrs = ["NORM","PREFIX","SUFFIX","SHAPE"]
rows = [5000,2500,2500,2500]
include_static_vectors = true

[components.ner.model.tok2vec.encode]
@architectures = "spacy.MaxoutWindowEncoder.v2"
width = 96
depth = 4
window_size = 1
maxout_pieces = 3

[components.tok2vec]
source = "pt_core_news_lg"

[components.tok2vec.model]
@architectures = "spacy.Tok2Vec.v2"

[components.tok2vec.model.embed]
@architectures = "spacy.MultiHashEmbed.v2"
width = 96
attrs = ["NORM","PREFIX","SUFFIX","SHAPE"]
rows = [5000,2500,2500,2500]
include_static_vectors = true

[components.tok2vec.model.encode]
@architectures = "spacy.MaxoutWindowEncoder.v2"
width = 96
depth = 4
window_size = 1
maxout_pieces = 3

[corpora]

[corpora.dev]
@readers = "spacy.Corpus.v1"
path = ${paths.dev}
max_length = 0
gold_preproc = false
limit = 0
augmenter = null

[corpora.train]
@readers = "spacy.Corpus.v1"
path = ${paths.train}
max_length = 0
gold_preproc = false
limit = 0
augmenter = null

[training]
dev_corpus = "corpora.dev"
train_corpus = "corpora.train"
seed = ${system.seed}
gpu_allocator = ${system.gpu_allocator}
dropout = 0.35
accumulate_gradient = 1
patience = 1600
max_epochs = 0
max_steps = 20000
eval_frequency = 200
frozen_components = []
annotating_components = []
before_to_disk = null

[training.batcher]
@batchers = "spacy.batch_by_words.v1"
discard_oversize = false
tolerance = 0.2
get_length = null

[training.batcher.size]
@schedules = "compounding.v1"
start = 100
stop = 1000
compound = 1.001
t = 0.0

[training.logger]
@loggers = "spacy.ConsoleLogger.v1"
progress_bar = false

[training.optimizer]
@optimizers = "Adam.v1"
beta1 = 0.9
beta2 = 0.999
L2_is_weight_decay = true
L2 = 0.01
grad_clip = 1.0
use_averages = false
eps = 0.00000001
learn_rate = 0.001

[training.score_weights]
pos_acc = null
morph_acc = null
morph_per_feat = null
dep_uas = null
dep_las = null
dep_las_per_type = null
sents_p = null
sents_r = null
sents_f = null
lemma_acc = null
ents_f = 1.0
ents_p = 0.0
ents_r = 0.0
ents_per_type = null

[pretraining]

[initialize]
vectors = "pt_core_news_lg"
init_tok2vec = ${paths.init_tok2vec}
vocab_data = null
lookups = null
before_init = null
after_init = null

[initialize.components]

[initialize.tokenizer]

As I mentioned, I've fiddled a bit with sourcing from a pre-trained model and tried to use a tok2vecListener. The sourcing part I based myself off mostly from this answer from StackOverflow, which I have no idea if it's correct at all.

Thanks a lot for all the help you've been giving me! This community is simply amazing :)

[adrianeboyd]

If you're trying to override some settings while using source, this is more of a missing feature. Currently source ignores any additional settings from your original config and just takes all the settings for new pipeline from the source model. This is currently on our internal to-do list, but I don't know when it will be available.

It's normal that after the component has been sourced, the config gets replaced with the full config from the source model. The source option is never in the exported config once a pipeline has been trained or assembled. It's just used in the initial config before the pipeline is initialized. The final saved config always contains the full explicit config settings needed to load that pipeline and there's no longer any link to the source model. This means the user loading your new pipeline doesn't need to have pt_core_news_lg installed -- all the settings and weights are copied into the new pipeline.