N L P Tasks

Natural Language Processing

 

 

 

Evaluation Results

Participating teams 

Four teams participated to the SeeDev-binary task and submitted 6 runs: 

  • MIC-CIS (1 run)
  • YNUBY (2 runs)
  • YNU-junyi (1 run)
  • Yunnan University 1510 (2 runs) 

 

Evaluation 

You may check and evaluate your predictions: 

  • on the training and development sets with the evaluation software;
  • on the test set with the online evaluation service. 

The evaluation is described on the BioNLP-ST 2016 SeeDev page. 

You can download all the charts and tables shown below. 

 

Global results 

Here are the global results for each run expressed in F1, Recall and Precision. 

The confidence interval has been obtained by bootstrap resampling (n=100).

Global Results

 

 
 
 
F1 by relation type

Results by relation type

Each axis represents a different type of relation.
Recall by relation typePrecision by relation type
F1 by type cluster

Relations by type cluster

Relation types have been gathered by cluster of similar types, thus reducing the number of categories of relations.

  • Comparison: Is_Functionally_Equivalent_To, Has_Sequence_Identical_To
  • Interaction: Interacts_With, Binds_To
  • Composition_Membership: Composes_Primary_Structure, Composes_Protein_Complex, Is_Protein_Domain_Of, Is_Member_Of_Family, Has_Sequence_Identical_To
  • Genic_Regulation: Regulates_Accumulation, Regulates_Expression, Regulates_Molecule_Activity, Binds_To, Interacts_With
  • Regulation: Regulates_Accumulation, Regulates_Development_Phase, Regulates_Expression, Regulates_Molecule_Activity, Regulates_Process, Regulates_Tissue_Development
  • Function: Is_Involved_In_Process, Transcribes_Or_Translates_To, Is_Functionally_Equivalent_To
Recall by type clusterPrecision by type cluster

 

Ignoring relation types and direction

The tick on each bar indicates the gain compared to the global results.

Ignoring relation type and direction