Two roots in a single sentence

[kanayer]

Hello, I am working on the Korean spaCy model and I run into the following issue. Could you please help me solve it?

Our team is developing a dataset for Korean and we put root as a deprel tag in case head is 0. Since that change was made, whenever I train spaCy transformer using ud_benchmark pipeline, displaCy keeps showing two roots for the compound Korean verbs like "하고 싶다" (want to do) or "먹고 있다" (eating as in the present continuous tense). I was wondering was it a mistake to mark words (mostly verbs) that have head 0 as root? Or is there any other explanation for this behavior?

The training dataset has 17719 docs and dev and test datasets have 2215 docs respectively.

Also, on a separate note, when I run python -m spacy project run train-transformer I get the following error:

('S', 0, 9000.0) 
('L-advmod', 0,9000.0)
('L-nsubj', 0, 9000.0)
('L-obj', 0, 9000.0)
('L-obl', 0, 9000.0)
('L-discourse', 0, 9000.0)
('L-xcomp', 0, 9000.0) 
('L-acl', 0, 9000.0)
('L-compound', 0, 9000.0)
('L-det', 0, 9000.0)
('L-ccomp', 0, 9000.0)
('L-cc', 0, 9000.0)
('L-amod', 0, 9000.0)
('L-dep', 0, 9000.0)
('L-nmod', 0, 9000.0)
('L-conj', 0, 9000.0)
('L-nummod', 0, 9000.0)
('L-nmod:poss', 0, 9000.0)
('L-aux', 0, 9000.0)
('L-advcl', 0, 9000.0)
('L-dislocated', 0, 9000.0)
('L-csubj', 0, 9000.0)
('L-amod:neg', 0, 9000.0)
('L-compound:svc', 0, 9000.0)
('L-advmod:neg', 0, 9000.0)
('L-aux:neg', 0, 9000.0)
('L-acl:neg', 0, 9000.0)
('L-xcomp:neg', 0, 9000.0)
('L-conj:neg', 0, 9000.0)
('L-ccomp:neg', 0, 9000.0)
('L-flat', 0, 9000.0)
('L-advmod:emph', 0, 9000.0)
('L-advcl:neg', 0, 9000.0)
('L-vocative', 0, 9000.0)
('L-mark', 0, 9000.0)
('L-dislocated:neg', 0, 9000.0)
('R-dep', 0, 9000.0)
('B-ROOT', 0, 9000.0)
('Gold sent starts?', 0, 0)   
Aborting and saving the final best model. Encountered exception:                  
ValueError('Could not find gold transition - see logs above.') 

The error was gone once I changed min_action_freq from 5 to 1. Do you think it is the reason why the trained model predicts two roots for a single sentence?

[adrianeboyd]

What does spacy debug data -V show for the dependency parser?

That error is usually related to having very rare labels or some other kind of error in your training data. The parser shouldn't ever predict two roots for the same sentence, where the root is the head of the dependency tree graph itself, ignoring the labels.

This shouldn't usually happen (it should be converted automatically in most cases), but maybe you've ended up with both ROOT and root in your training data somehow? (At what point are you editing the dependency labels for this?)

[kanayer]

The first time I run the debugger it gave me the following output:

Turns out, there were two cases in the dataset when det was the deprel tag even though the head was 0. I have updated the deprel tags but I wonder how come only two cases could affect the model to the extent where it constantly predicts multiple roots? Is it possible?

After correcting the error, the debugger gives the following error:

Could you please help me interpret what does this error message means? I have searched but I don't have the deprel tag "advmod||ROOT" in my dataset, so I am not sure where does this error comes from.

[adrianeboyd]

The dependency trees are projectivized because the parser algorithm can only predict projective trees. The projective versions uses these extended dependency labels to mark the places that were modified, and then they're deprojectivized (modifying the trees and removing these marks) before being saved in the doc. Links to details about the algorithms at the top here: https://spacy.io/api/dependencyparser

I think the easiest solution here is to fix that one non-projective tree. You can use is_nonproj_tree from spacy.pipeline._parser_internals.nonproj to check, see:

spaCy/spacy/cli/debug_data.py

Lines 726 to 736 in 42072f4

	if "parser" in factory_names:
	aligned_heads, aligned_deps = eg.get_aligned_parse(projectivize=make_proj)
	data["deps"].update([x for x in aligned_deps if x is not None])
	for i, (dep, head) in enumerate(zip(aligned_deps, aligned_heads)):
	if head == i:
	data["roots"].update([dep])
	data["n_sents"] += 1
	if nonproj.is_nonproj_tree(aligned_heads):
	data["n_nonproj"] += 1
	if nonproj.contains_cycle(aligned_heads):
	data["n_cycles"] += 1

We don't provide any scripts that point out the exact spot in the corpus, but you can add in some temporary debugging in this section of debug data to figure out which sentence it is.

Still: it should not be possible for the dependency parses to have multiple roots. Are you sure that's what's going on? It's possible that the parser will predict two separate sentences where your training data has one sentence, but there shouldn't be multiple roots within one predicted sentence.

[kanayer]

Here is an example of the analysis made by the trained Korean spaCy transformer model. As you can see there are two different ROOTs for one sentence. Or is it predicting two separate sentences for one sentence?

Could you please explain in more detail how exactly should I use is_nonproj_tree to fix the one non-projective tree? Do I provide data to the script you've shared as an input, or do I import spacy.pipeline._parser_internals.nonproj as the library or do I call it as the command in the terminal?

[adrianeboyd]

We don't provide any scripts for doing this. It would look something like this (and then use displacy or whatever you'd like to show the parse and find the crossing arc):

import spacy
from spacy.tokens import DocBin
from spacy.pipeline._parser_internals.nonproj import is_nonproj_tree

nlp = spacy.blank("xx")  # the lang/tokenizer config doesn't matter for this
docbin = DocBin().from_disk("train.spacy")
for doc in docbin.get_docs(nlp.vocab):
    if is_nonproj_tree([t.head.i for t in doc]):
        print(doc)

This isn't multiple roots, this is the parser predicting multiple sentences. Typically the parser does this because there were similar sentences in the training data. The parser is pretty good at splitting sentences on phrase boundaries (like for headlines or run-on sentences), but depending on the training data it can learn to split too often or not often enough. I would suggest that you try varying the text lengths and number/types of sentences a bit more in the training docs and see if the results improve, but the main thing is that the training data should look like the data you have in practice, and you want to have docs with multiple sentences or it won't learn to split sentences at all. It might also make it easier for the parser to identify sentence boundaries if the punctuation is split as a separate token, but you'd have to modify all the training data to do that.

Alternatively, if you know that you have one sentence in advance or know where the boundaries are, you can set token.is_sent_start = True/False in advance (with a custom component, sentencizer, senter, etc.) and then the parser will use those sentence boundaries.

[ivan-kleshnin]

Is there some built-in helper to merge multiple sentences for Example.from_dict(nlp.make_doc(doc), gold) kind of line?

spacy convert command has the n-sents flag but I can't use spacy convert directly. It does not support mixed formats and my corpus is partially CONLL-U, partially JSON.

I currently do this manually (contatenating texts, tags, deps and reindexed heads for each new added sentence) but it feels hacky.